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PREFACK 

This two-volume .\fanuul provides information employment and trainina 
agencies can use in 

1. developing and operating client assessment programs (Volume I); and 

2. identifying, adapting or developing special assessment techniques for 
se\er^ly disadvantaged clients (tor whom widely available standard assess- 
nient tools, such as the General Aptitude Test Battery (GATB], may be 
inappropriate) (Volume II). 

Although the two volumes are meant to be used together, they may in some 
cases have independent value, e.g.. for planning an assessment program for 
which the specitic techniques have already been identified or for a research use 
ot assessment techniques where application in a practitioner setting is not 
required. 

Volume I: Introduction to A.ssessment Program Develop-nent has three sec- 
tions: an introduaum describing recent trends in client assessment for employ- 
ment and training agencies, presenting some background about the Manual 
and giving some basic concepts about client assessment; a section on current 
praanes in employment and training agencies, including some detailed 
descriptions ol existing assessment programs; and a discussion of assessmenf 
promim development, including a conceptual model of assessment and a plan- 
ning checklist for program developers. Volume I also contains an appendix 
listing intormatioii resources for assessment program development. 

Volume II: Assessment Techniques begins with a brief review of standard 
assessment techniques, such as the GATB and Strong-Campbell Interest Inven- 
tory, devices now widely used in employment and training agencies. Then a 
catalog IS given of special assessment techniques, each entered in capsule form 
using a standard description format. Volume II also contains two appendices: 
(A) a chart listing special devices available for assessment of seriously disad- 
vantaged persons; and (B) information resources regarding assessment of the 
disabled. 

I his two-volume Manual is designed principally for use by employment and 
training agcncv persnnnel responsible for developing and operating assessment 
programs; other agency staff involved with these programs; and administra- 
tors of local agencies, who have to make the larger decisions about assessment 
:ind hiiw It should tit into the rest of their service delivery operation. The Man- 
ual may be of secondary, but in some cases important, interest to researchers 
and policv makers and also to persons concerned with assessment in other ser- 
vice ddiverv settings (e.g.. vocational rehabilitation agencies). It should be of 
direct value to persons working in client assessment programs for employment 
and training services that arc attached to educational institutions or agencies. 
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Setting the Stage 

emnTw"^"" challeni^es have emerged recently regarding client asssessment in 
r« rc " ' particular, the 1978 legislation reauthor- 

i/«ng the Comprehensive Employmem and Training Act (CETA) contains two 

^i^Zn rTerrf'^'r^'^r"" activities r^r Tfe 

ally supported adult and youth programs. Section 205. under Title U. Partici- 

pan Assessment, requires each Prime Sponsor to develop a personalized 

employab.l,ty plan . .taking imo consideration an indivuLl's skTlls tnter 

es s and career objectives . . .and shall consider the barriers to employment or 

advancement faced by that individual. . The section goes on tnar"'an 

assessment of appropriate training and supportive services shall be made at 

I h ' ' f rr"" ^^'^"^ ^^^'^^^d periodically. . .(and 

shall be included m each individual's employability plan." Section 445 the 
generaPprovisions of the Title IV youth programs.^equires "app^^p'ri 

or foL """"I" '''''''' '^'^^^^^ tranSn o 

youth, .to permanent jobs. . .or education or training programs." 

assIJs"ncnt 1' ITr'"^ '^'^^'^^'^'^^ of systematic client 
nm no^^^^^ h r r h ^•'"P'^^^'"^^*"^^"^ '^^'"'"8 -^^rvice delivery. Agencies that 
n ay nc cr before have considered having a formal assessment program now 
I. cly to do so because of this part of the new legislation. Almos! alUgen 

^Z!r^ i^'^'^'^'r' '^'^^''"^ "'^'^'^ improve-their in-house or 
subtontra;tcd assessment functions. 

rh^^w'ST-^ T"^ 'T'f:"^ assessment that is gathering force is represented by 
irwh . / • /T ""^"'r' Procedures (as published 

. he tederal Re,,ster, Vol. 43. No. 166). These Guidelines set out a revised 
and expanded pohcy one that applies to employment and training agencies as 
well as to private industry, for assuring that «sse.ssmem devices u.sed to make 
decisions about enipKninent or employment-related activities do not unfairly 
d.s.Tim.Mate against certain protected groups, such as .omen and racial or 
c ic ,inMor.ties. I-nforcenien, of these Guidelines wil, compel agencies to 
look at their assessment procedures in new ways, probably to revise or elim- 
.nate some met ods „ow being used because of potential or actual discrimina- 
M. arul to gather data on validity and possible adverse impact. Agencies not 
u u ertaking such steps may fmd themselves subject to administrative disc^I 
plinarv action, or even to lawsuits alleging discrimination 

I-.nallv there is a challenge presented by assessmem technology itself On 
the o„, hand, neu approaches to assessment of seriously disadvantaged per- 
sons have been developed over the last 10 years, and some of these are 



being used in local employment and training settings. Developments from the 
United States Employment Scrvivre, from research funded by the Department 
of Labor and from the private sector figure in this advancing state of the art of 
assessment technology for persons who cannot appropriately be assessed using 
the standard paper-and-pencil tests. But the very availability of this new 
resource for assessment creates problems in terms of selecting which assess- 
ment techniques would be most appropriate for use in a given setting, and in 
adap(in}i available assessment technology to varying'local needs. 

Moreover, there is evidence that in a significant number of employment and 
training agency settings, mapprv^priatc psychological tests are now being used 
in client assessment. There arc many *'horror stories' —use of sophisticated 
clinical projective tests, such as the Rorschach, for making employment- 
related decisions with the tests administered and scored by minimally trained 
staff; use of gross measures of psychopathology, such as the Mif}^iesota Multi- 
phasic Personahiy Inventorw for making determinations about individuals' 
personality functioning in tlie normal range* Whether the problem is a test 
administered for a purpose it was never intended to serve or inadequate train- 
ing of testing personnel, there is substantial reason :o believe that assessment 
in sonic employnient and training agency settings may be off on unproductive, 
even poientiall> harmful, tangents. The harm to clients mi^y come from pro- 
Mding maomirate. misleading information about their employment-related 
charMCieriNiics; harm to agencies may be in terms of potemial legal liability for 
decisions made on the basis of such information. 

These are some of the important challenges to CETA-funded employment 
and training agencies— a press on the one hand to assess, to improve service 
delivery through appropriate use of assessment results; and a press on the 
other hand not to use given approaches to assessment unless it is established 
that the information they provide is relevant and that clients can perform 
validly on them. Finally, there is the problem of assessing the resources that do 
exist for helping to meet these needs. 
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IhlVc-in serV'".' ^-^'7^"-"/'^* ^^'^'^"^'^ ''^^^'^^ these challenges 

P ovl^^ a?;,^ ^''^^'""'^ "^^^'^-'-^ 'or local 

. Tr agencies, starting with the conceptuali/ation of the 

I he entire Manual is designed to speak most directly to employment and 
training agency staff in charge of developing or maintaining cS asTsm^^^^ 
programs. Practitioners who have to live with anH ..... .h^ T assessment 

u o o,hor a..pa-,s or ,hdr agency's opera.io,,., are .he .wo oZmos impo" 
tan. aud,e,uo. lor rh. Mamal. I, may al«„ be of some in.eres. .res. deve 
opors. researchers and policy „,akers a. .he Federal level 

Ihe presen. volumes represent a subs.an.ial revision and updale of a work 

u..o.her ,W ,he „rs. ,in,e availark'nlirah™ ' rh^Tstfand::?:; 

X.:! ,. onT7;,''.h:.'r'"'* °' with disadvamaged 

Popula.ions. Included were .nitrumen.s developed by .he Depanmen, nf 

u:d:,:?j:'in;:;sMh ^JVam^^T h 

<«« m, r i • ■ i""'"! publicaiion. some 

15 (X«. copies 01 ,ha. monograph have bcH..n dis.riba.ed. reflecling a rea neTd 

However, n.uch has happened since 1972. The Comprehensive Employmen. 

uwinuKU and multiservice general agencies at the local level are the rule- 
despite reduced R&D support for assessment from the Departmem of Labo * 
soinc. new assessment devices have emerged since the im monograph wai 

aiangcu. I UK lausuits have become prevalent in connection with allegedlv 
d s rnninatory assessment or selection procedures in private indust y and 
u Its are being or mav be mounted against emplovmcnt and tra.lg agendes 
C ongressional attention has focused on as.scssment practices in ^ocai 
C I lA-tundcd agencies, and not always with favor 
l or all of these reasons, a new manual on client as.sessment is in order 

Z: t":: --t'nement of the autho 's 1972 

work. Uure is a kev emphasis on assessment program devehpmem~on the 
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practicalities of ihinkins about, planning for and iniplementing the system 
that will make possible the use of selected assessment devices. However, ^hile 
the first monograph was primarily a state-of-the-art review designed to 
describe Department of Labor-funded research projects in the assessment 
area, the present effort has a broader goal. It is intended to be a practical guide 
book for use by local program personnel at the several levels mentioned above, 
I'uriher, more attention has been given in this volume to resource identifica- 
tion, so that the \tanual can serve as an *'on-paper clearinghouse" for infor- 
mation about assessment techniques for the seriously disadvantaged. 
Wherever possible, information is provided on how to acquire a copy of what- 
ever is being discussed. 

l-mphasis on the practical information needs of assessment program devel- 
opers in employment and training agencies also is retlecied^in the strategies 
used lo develop these volumes. For the W72 monograph, a key developmental 
technique was a one-day conference whose participants were mostly assess- 
ment technique developers and researchers, with only a few practitioners 
present. Here, however, most of the collaboration has been with program 
developers. 

This collaboration has proceeded m two ways: first, in the review of drafts 
of this numuscripi by a number of persons from employment and training 
agencies: and second, via presentations to employment and training agency 
stall in the Orange t ouniv, California area given by the author during the 
project's initial stages. .\{ these presentations, outlines of the Manual were cir- 
culated, and there was considerable audience discussion of what ought to be 
included in the Manual to make it most relevant to assessment program 
devciopers, practitioners and administrators in local agencies. In essence, 
then, (lienf As.wwment: A Manual for Employment and Training Agencies is 
designed for p'^actitioners. 



What s Included— and What Isn't 

In discussing special assessm^rni techniques here, severely disadvantaged per- 
sons are detined as those individuals whose sociocultural or educational defi- 
cits inierfere with maximizing their employability potential. More specifically, 
the scverelv disadvantaged are those persons who cannot validly complete 
standard cheni assessment techniques (e.g., the General Aptitude Test Battery) 
because of their sociocultural or educational deficits, including prior lack of 
expenciicc or aversive experience with testing. This definition includes a large 
number ot persons receiving or eligible for employment and training services. 

I he dctinuivM) excludes persons whose primary employability deficit has to 
do with such factors as phvsical disability or mental illness. Assessment devices 



aie available tor these groups, and currently many more such clients are com- 
ing 10 C'l.IA-spoiisored employment and training agencies to receive counsel- 
ing, iraimng and placement servic-s. However. Hmits in the resources support- 
ing this work necessitated restricting the focus to assessment devices tor ttie 
sesercK disadvantaged as defined above. Nevertheless, what is said here about 
how to develop, implement and maintain assessment programs in employment 
and training agencies is nor limited in any way. Kxcept for the coverage of 
assc-sinent devices in the second volume, this .\iatwul is intended to be used in 
setting up and operating anv kind of assessment program in an employment 
and training settjfg. 

Standard tools, such as the General Aptitude Test Batterv and other U S 
rniploymcm Service-published assessment devices designed primarily for non- 
disadvantaged persons, are treated only in brief because they already have 
been well covered in other publications, e.g.. those of the Employment Ser- 
vice. Information about other standard tests that might be u.s< d in an employ- 
ment and training agencv setting can be obtained through reference to retrieval 
resvnuccs m the sccoikI volume and the appendix of this volume. Resources for 
asscs^mcnt of jihvsicall> disabled persons are identified in Appendix B of live 
second volunicf 

^ Specific information about the contents of this Manual is provided in the 
Pretace. I he next Neciion offers a fairly comprehensive definition of assess- 
ment as hackuround tor a discussion of assessment program development and 
operation. 

Definiiion of Assessment 

Assessment involves j-atheniiv: information about the aptitudes, skills, atti- 
tudes, personahtv traits and life and work histories of persons seeking services 
trom local oniiMovment and training- agencies. Results of assessment may be 
Used to meet one o\ more of the objectives stated below. 

Purposes of .Assessment 

I I orailnttiimu dicnis h> niM'n scn icf rroantms or proaram eU-menis. Ciiven 
cciiam pioiirain L-oaU. n mav be necessarv to restrict the flow of clients to 
those lueeiinL- a-ttam cnieiia. l or example, in a training prouram where 
onlv iiiose with ai IcaM sixih-gradc math skills are likely to benefit, assess- 

•llu-u .. .1, -otl..^ .iii'p.i.-.- -n ilu- ..K.u-.K-liniii..n ..t •.^lMr^ .iivliulai .iii.l «lui imi'i lor 

v.iM'l'-. "I.- -W.M.J ...iimv- .K-w,..s .iiioiiu.n r.. .1 n.miKi ,.| .ish-smikmii sVsUiu^ or Jo^-ko 
.1. W.'iv.l I'l .11... -i^ ., |.(u....,lU ^hs.ihlal i>o|nil.)ii.Mi 
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mem might be used lo screen out prospective enrollees with less than the 
required math ability. Presumably those excluded would be channeled to 
another program or given the remedial education necessary to achieve a 
sixth-grade math level. In other cases, limiteu program resources mean that 
only a certain number of clients can be accommodated at one time. Thus, it 
becomes necessary to restrict the flow of those entering the program. 
Assessment might then be used to select those most likely to succeed. 

In cither case» it is important that the variables assessed are truly relevant 
to successful outcomes and are measured as accurately as po^sible. Assess- 
ment should not be a means simply of keeping a service effort io a manage- 
able si/e; random selection would achieve this goal just as well. Use of 
assessment tools should reflect a commitment to provide the best possible 
services to program applicants. On the other hand, even accurate selection 
of those most likely to succeed, e.g., in a training program, might be chal- 
lenged on the grounds that it may discriminate against those who need the 
program's services most. Clear statements of program goals are required, 
therefore, in order to decide how assessment can best be used for admission 
decision.N. 

2. For assignment lo training. For many disadvantaged individuals, employ- 
ability development entails acquiring vocational and behavioral skills nec- 
essary for successful work functioning. Perhaps the key variable here is 
••learning potential*' or •*trainability," the individual's capacity for behav- 
ioral change following a training experience— a variable that systematic 
assessment can help to measure. A crucial consideration is the number of 
training options open. If only two training programs are available, it may 
not take much information to decide which is preferable for given enroll- 
ees. Also, there is some research evidence to suggest that characteristics 
associated with training success are not necessarily related to later job suc- 
cess. Several phases of assessment then may be required to guide an enroll- 
ee from entry through training to placement, each one clearly spelled out as 
to how results will be used. 

3. For counseling and personal development. At least as important as skill 
training is preparing clients psychologically for the world of work. Such 
vocational and personal exploration ideally is done by a counselor with 
rather than to or for the client. If so, assessment data must be relevant and 
understandable to the person counseled. 

Almost any valid assessment tool can be used as part of a growth experi- 
ence for the client by the simple mechanism of feeding back results in terms 
he or she will understand. Feedback may offer the client an opportunity to 
participate in decision-making processes. Information about personal or 
vocational characteristics and about available training or placement 



options can be given. Then the client can be encouraged to work with the 
counselor or eniployability development team to formulate a plan for his or 
her own vocational advancement. To a disadvantaged person whose previ- 
ous hte experience may have involved maiiy decisions or outcomes imposed 
trom outside, such a participative experience can be extremely valuable in 

. l or job nlacement. tiiven accurate assessment of job characteristics, suc- 
cessful placement involves matching the individual to a suitable job open- 
•ng. It is crucial that assessment devices tap variables of real importance to 
job success. I hus. employment and training agency staff need to know how 
well a device predicts success, and under what conditions, when using it to 
help make placement decisions. 

f or pretesting orieniatum. Sometimes performance on sub.sequent assess- 
ment tasks can be enhanced when people who are not very ''test wise" (or 
who are very anxious about being assessed) are given a chance to try out 
their test-taking skills. When such trial runs are given, it i.s important that 
the practice materials bear some resemblance to the test device.": that will be 
used later, w.thout reducing the validity of results because of practice 
ctfects. Practice with test-like materials can be imporfint both for skill 
development in test taking and for generating more psychological comfort 
during assessment. 

tor e lution of the employment and training agency. At a time when the 
public and Its representatives in legislatures are demanding justification of 
government expenditures on all levels, .systematic evaluations of CETA- 
sponsored agency effectiveness are increasingly frequent. Some of the 
problems mhereiu in establishing appropriate criteria to judge program 
success will be discussed later in this volume. When these criteria involve 
assessing enrollees. e.g.. to find out what they've learned or how they've 
changed during their participation in a program, it may be particularly 
tmportant that assessment tools used at entry permit valid re-administra- 
tion and that change scores be translatable into concrete conclu.sions about 
ftrogram tunctioning. 

7. f-ortestd. •■'Vv/»t7,^</m/rt^svar(•/^ local employnient and training agency 
clients sometimes serve as subjects in basic research efforts. Understanding 
ot the impact of cultural disadvantagement on vocational adjustment can 
be increased through empirical studies of clients' life histories and psycho- 
.ogical characteristics. Clients serving as subjects tor such studies should 
represent an adequate sample ot the overall target population. Detrimental 
ctfects on clients as a result of participation must be avoided. For example 
since academic tests represent highly aversive experiences for n any of the 
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disadvantaged, research devices should be carefully tailored to minimi/e 
anxiety and other negative reactions. 

Putting Assessment in C onte\t: Some (leneral Criteria 

What constitutes a successful assessment program? The following criteria are 
one anssver to this question and should be kept in mind as readers consider the 
information and guidance offered in the rest of this volume. 

1 . A wessmem shouUi be relevant to the needs and priorities of the local aaen- 
ry settuii^ in which it is used and should fit into the total context of agency 
objectives. Assessment is only one of many services a local employment 
and training agenvy svill provide. Lven the smallest and simplest service ef- 
fort may entail a number of program components with which the assess- 
ment effort must fit reasonably well. In designing an assessment program, 
therefore, questions like the following become relevant: When should 
assessment occur? How fast must the results of assessment be delivered in 
order ti> prove useful for making decisions? How does the cost (in money, 
manpower and physical space) for asssessn^ent relate to the other program 
components? What possible side effects of as.sessment mjght be expected to 
impact on other service delivery components? 

( )nly bv pavmg careful attention to issues such as these can an assessment 
program be ciMistructed to fit smoothly with the entire service delivery 
operation. Budgets and operating constraints are just as important for 
aNNCNNmeni-program developers as for those developing a training program, 
a loh'placeineni effort, and so fortli. The context into which assessment 
tus in a local emplovment and training agency will be discussed further in 
Section II. 

2. Fach individual asseysment device must serve a practical purpose in achiev- 
//K' emplovment and training agency goals for given clients. Evaluation o** 
the etfectivenesN of assessment techniques or entire programs may thus be 
measured along one or more of the following factors: 

a. Soundness and relevance of assessment results for input to agency deci- 
NUMi making. 

b. Actual impact on service delivery (measured in terms of stated agency 
objectives to provide counseling, training or placement services). 

c. { Ost of the assessment technique or program (including cost of acquir- 
ing and implementing the assessment technology; staff time and facil- 
ities needed to operate the assessment program; amount of time it takes 
to Ci>mplete the assessment process). 



d. //mW/m-vs with which asscssincm results are provided in relation to 
other aspects ot the ser\ ice delivery operation. 

e. -U't-m V sw/y .u-if riumv ot the assessment proisrani and its results, 
t. ( ■//«'/// ucivricith\' of the assessment program and its results. 

g. t m'xnrn'd pavoj/s and wiiuinr side i'/fccis f rom assessment, such as 
delavs ,n other aspects o\ serv ice delivery because of the need to assess at 
a certain point in the ^er\ ice delivery process. 
Assessment, just as much as anv other program component in a local employ- 
ment and irainmg agencv . should be subjected to rigorous evaluation to deter- 
mine whethe- or not the effort should be continued and to ascertain what 
improvements could be made to uicrease overall effectiveness and efficiency of 
operation. Moie will be said later about evaluation of assessment programs 
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II. CURRENT PRACTICES 



Overview 

This section is designed lo serve three main purposes: first, to review infor- 
mally the kinds of assessment programs now operating in employment and 
training agencies; second, to identify some of the major problems and pitfalls 
now being faced in client assessment; and third, to present some ways in which 
these problems have been addressed. These subjects will be approached by pre- 
senting the results of two surveys (one national and one regional) of assess- 
ment practices, supplemented by observations based on the author*s own expe- 
rience, and by five original program descriptions especially prepared for this 
volume. 



Portrait of Client Assessment: The Results of Two Surveys 

While many data-gathering efforts sponsored by the Department of Labor or 
by local prime sponsors have touched on the topic of assessment, to the 
author's knovv ledge there are only two recent surveys focusing on this subject: 
a 1976 report by Mark Battle Associates on assessment practices among CETA 
prime sponsors and a 1979 study by A.L. Nellum and Associates about coun- 
seling and assessment practices in DOL Region 111* Results of these two 
survey studies are summarized below. 

The Mark Battle surve>* In late 1975, the Department of Labor's Employment 
and Training Administration rei:eived a letter from the Justice Department 
expressmg con:ern that CETA prime sponsors might be administering or using 
tests in a manner that would Interfere with equal opportunity rights of CETA 
participants. To determine the possible validity of ihis concern, DOL devel- 
oped and administered a questionnaire for CETA prime sponsors and their 
subagents lo complete regarding their use of test instruments in CETA pro- 
grams. Mark Battle Associates was requested to analy/e data from these ques- 
tionnaires and compare findings with the available literature on psychological 
testing io deiermine how appropriately these instruments were being used in 
Cl\T\ prime sponsor and subageni assc smeni programs. The study's final 
report was submiiied in 1976(Mark Battle Associates, 1976). 

*c opios o\ ihcM" rcporis ma\ bo obiaincd bs \^rUing to Mark Battle AsMKiaics. lOl*^ 19th Strtct, 
N VV . VVashirijiion. I) ( 2iK)^6. and lo A. I . Nellum and Associates. 161 lorbes Sircei, Urain- 
trec. \laNsavhu>enNl)2lH4 
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An ovcrMcH at findings from the survey data and the testing literature and 
three primary recommendations for further work in this area have been 
excerpted triwn the Battle Associates report and are presented here. 
Hndin|{s from the surve> data 

• There v.cu: 2.685 C'iilA prime sponsors and subagents resDonding to 'he 
sursev. 



• I here vNcrc M\ different tests used by respondents, including USES tests. 

• Ihc most popular USl-S test was the General Aptitude Test Battery; second 
most popular was the Clerical Skills test. 

• Sixty percent (W'n) of CEtTA prime sponsors/subagems responding to the 
survey used no lesis in their activities. 

• iHcntN-three percent {2}^) of ChTA prime spon.sors/subagents using tests 
utili/eJ USliS-de\ eloped tests. 

• Thirty -five percent OS'^'o) wore CETA prime spon.sors/subagents used 
'V»//jfr" tests than those who utilized USfcS-developed tests. 

• l our percent '4«o) of CV.TA prime sponsors/subagents used both USES- 
developed and other tests. 

• ()nl> 16 percent of the respondents indicated that they contracted with the 
Sl-SA for testing services; only one-third of the.se had been given training in 
the interpretation and use of the test results. 

• l ifty-nine percent {59<^o) of respondems did not respond to the request for 
intormation regarding the purpose for which the tests were being u.sed, 

• I he most popular purpose for tests was to assess academic and/or basic 
skills and tor other diagnostic and evaluative purposes (29%). 

FindinKs from testing literature 

• or the .^21 tests used. 1% were not found in the testing literature;* 125 were 
reviewed and are discussed in the report. 

• Other than miscellaneous tests, which were larpely unidentified in the litera- 
ture, the most CrequentK used tests were achievement batteries ;cited 222 
times), followed b> vocation-interest tests (cited 155 times). 

• OtiK seven percent (70o) of the other tests used by respondents were multi- 
apiiiude test hatieries. such as OATB. 

• t tchtccn percent (!S-»o) of tests being used by C'ETA prime sponsor.s/sub- 
ageiit^ wcie valid for specific occupations. 



•|( >M..uM K- M.-u-d th... H.,..l. \vM..MU-^' .K.ovv lo ilK- u-si.nv. hu'r.m,ro ..pnarcniK Nvas 

u-iv i.mn. J. M..i,u „„pot,.,„, r.i.-u-.KCv.iro noi moni.onoU ai .,|| mihcir ropori. Ihus. this find- 
nii: lml^l hi- pnu-ipri-u-d wiih iitc.ii v.iiiii.ui. 
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• Sixty-nine percent (69^'o) of tests being used by ChJA prime sponsors/siib- 
agenis were not valid tor v^omen. 

• Seventy percent OO^-n) of tests being used by C'liTA prime sponsors/sub- 
agents were not valid for niinorities. 

• Seventy-four percent (74»^o) of the tests reviewed were designed to be admin- 
istered by a counselor; 14 percent should be administered by a psychologist 
or other trained professional 

• Twenty-six t?sts were m the Character and Personality category. Of these, 
six employ proiective techniques and, therefore, have questionable value as 
instruments for C t lA purposes. 

Recommendktions 

1. Data provided by the subject survey are not reliable or valiu enough to be 
conclusive regarding usage of tests by CETA prime sponsors and sub- 
agents. Therefore, it is recommended that further in-depth research be con- 
ducted if generalization to the population of CETA prime sponsors and 
subagems is desired. 

2. If another survey is conducted, the survey questionnaire should be designed 
to elicit reliable results (i.e., more questions will be needed to tap the uni- 
verse of possible responses, and a higher question-by-quesiion respon.se 
rate will be required.) 

3. Cluidelines for future use of tests for the CETA program should be devel- 
oped using more reliable data. DOL/ETA should consider alternative 
methods for collectiiig needed information, such as site visits to a sample of 
prime sponsors anu subagents. Case studies would be one suggested 
approach (pp. 5-6). 

The \A.. Nellum Survej in Region III. In 1978, A. L. Nellum and Associates 
was contracted by the Departitient of Labor's Region III Regional Office to 
appraise vocational counseling delivery systems operating among the CETA 
prime sponsors in Region ill, to design a model system for the delivery of 
vocational counseling services and to provide technical assistance and training 
. in the implementation of this system. A systematic assessment function ^as to 
be a part of the system. 

In pursuit of the first objective, a questionnaire survey (augmented by site 
visits leading to case studies in selected prime spon.sor agencies/ was conducted 
in late 1978. Some of the most important conclu.sions drawn from the survey 
and case studies are as follows: 

• The majority of test and assessment activity among those Region III agen- 
cies respondnig to the survey is subcontracted to other agencies, most com- 
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monh state cmplo>iucnt services .but also including a variety of private 
organi/atioiis. 

• Airhough there is a vside variety of test instruments in use. the CJATB is the 
one most frequentls eniplosed by these agencies. 

' Assessment, like counseling services, is provided on an a.s-nceded basis 
lathcr than through a predeveloped strategy. 

' riie counselor is UMiallv the sole decision maker regarding client progres.s 
through the service delivery system, including the assessment component. 
Assessment sv stems differ widely from one prime sponsor to another and, in 
many cases, withm the area served by a given prime sponsor. 
Most of the tests used bv agencies in the region are paper-and-pencil instru- 
ments evidencing little sensitivity to cultural and linguistic differences 
among participants. 

Counselors are rarelv well trained in procedures of administering, scoring, 
and inter <reting assessment instruments or in using assessment results when 
the assessment function has been subcontracted. 



f-indings from the Nellum survey must be qualified by mentioning that 
fewer than 50 percent of the prime sponsors in Region ill responded to the 
questionnaire. This shortcoming was mitigated somewhat, however, by the 
case studies of eight fMA programs Nellum prepared. Although these case 
studies were not primariK focused on assessment, and typically included only 
two or three sentences of information about assessmem functions in the given 
employment and training agency, they did provide a good picture of as.sess- 
luent in the context of other service delivery functions. The Nellum report pro- 
vides the tollowmg summary of assessment activities in the eight prime spon- 
sors un which thev completed case studies (pp. 66-67); 

I Miailv .1 u-si IS .idrninisKTai .is an iiiiiial scra-ning device lo Ueierniine an appro- 
ptiaic iMi'ninv! .uhmiv tor a pariiapam. In eases vvhere- iIk- .lob Service is rcspoii- 
Mhlc tor .isscssn-eni . the CiAI H is ;ulniimsiereU. .lob Service counselors imer- 
Piei ihese result in .i.votdaiiee with ihc partieipaflt's educaiional background. 
acquiu-J skills .inJ intoresis. A reterral is then n.atie. 

lOiK-sUvj uscsiiK- R \I .uKl WR.M tor assess„u-ni of all panieipants. Based on 
ilie inuMl .in..| sdt .ulniiiied linuied results, a referral is u,aJe to a program aciiv- 
.i\ wlK-ic more in Jqnh u-siimv- is Joik-. The \ouih program . . . adminisiers ihe 
< M u. all ilk-ir pariKipanis. liooilwill Indiisirics performs in-denth vocational 
asscssniaii ot spceial nceds pariiopanis using the Hester l-vaUuiiioii System. In 
[.muilK-i .itvj. the 46 cmpknmeni and training subcontractors determine which 
lesi loadniinisia t^-.i their programs. I here is no vonsistaicy among urograms. 

Ml v>i ihc Pitme Spunsois admitted the difficuliv in using tests ihat were not 
lunmed tor ihe U I X populaium. J here is a need tor Jittereni tests tor youth 
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and dduhs, dif tcrcni tests to screen tor tKVupaiional training and academic prepa- 
ration. I he OK" in lone cii> I haN developed a special test to meet the needs of their 
program and participant population. A. . .subcontractor vihich refers the 
majoMty of its participants to a CiM) program uses the C AI. This is an appropri- 
ate test lor the academic progratn referral, but it still provides limited informa- 
tion about the participant's vtvaiional abilities and interests. 

Some of the more sophisticated tests were used in >outh programs. lOne pro- 
gram] evaluates their >outh participants with a series of instrument's: Jl:SNl:R 
Behavioral Scale, C AT, Independent Reading Inventory. lAnotherj uses the 
C'OAIS system as a pre- and posi-iesi measure of the job-seeking capabilities of 
the participants (16-1^ vear olds) and as the basis for a prescriptive counseling 
system. ;Aiu>ther program) uses the Strong-Campbell and the Career Maturity 
Inventory <C'MI) and the Holland SDS as tests of v^Kational interest. lA county 
program) has assimilated into its adult programs many of the test instruments 
developed and used in its vouth programs that it found helpful. 

'Standardized tests, despite these shorteomings, do return some measure of 
vocational interest and abiluv. and also an insight into th*? academic level of the 
individual participant being tested. The value of the.se tests to the counseling pro- 
cess IS dependent upon (1) the environment and procedures under which it is 
administered. \2) the degree and quality of counselor-participant interaction in 
the interpretation of the results, and (3) the sharing and communication of the test 
results with other agencies and programs to which the participant is referred. In 
many cases, u was observed that the test results were discussed only with the par- 
ticipant. The counselor used this discussion to make a decision concerning the 
participant's M)P referral and then the te.st results were filed in the participant's 
folder. 

It was observed that sometimes testing and assessment were conducted by pro- 
gram agents to achieve their contracted performance obligations rather than using 
these tools to assist both the participant and the counselor in the deci.sion-making 
process and in the development and finali/ation of the HDP. 

In summaiv, in all the sues visited, testing and as.sessmeiu were conducted in 
ci>niunction wnh a counseUu. The problems identified were as wide and as varied 
.IV 'he number ot tests used m Region 111: the overemphasis ol academically 
i>rienied instruments; the lack ot uniformity within Prime Sponsor jurisdiction; 
and the limited useot instruitieiits normcd to the I A client population. 

No maiter what the inNtrument or the procedure used, the test and assessment 
process shi>uid be incorporated into the larger counseling svsiem. .Ml l*rime Spon- 
sor s requested technical assistance and training i,i the use and interpretation of 
tests wuh CI I A participants. 

Assessment Program Descriptions 

The descriptions presented here grew out of an effort to learn what was hap- 
pening in local assessment programs in employment and training agencies in 
the southern California area. Contact was established with a consortium of 
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agoiKies iM Oianijc ( ountv. ( alifornia. ami the authtir presented several lee- 
tines to this LMoiip duritij! coiiterenees on assesstneiit: As a resuh of this con- 
tact. «n\itatioti> tor a site usit \sere exteitiled by H>\eral assessment program 
Vooidmators to the \snter aiu' ' i> statt. 

Ihese \isit> uhimatelv Here u>ed to ie\iev, each assessment projiram in 
tneetinjis uith appropriate statt in order to deii\e eiu-uuh information about 
the \\a\ the assessment ptoer.cm is oriiani/ed to prepare a very simple sum- 
mai\ about it. It also \sas deemed important to include a description of what 
the I S. I mpKnment SerMce is doing thiough its local ofj^s in each state via 
a lepott on the Calitoima State I niplovment Service. llie description of the 
Jack Hredin i ommunitv liismute \sas completed \Mthout a personal site visit 
but contains the same basic categories of information. 



Pk.h.k vm Si mi ,,i c vi ii..kM , I mpio^mi m |)i \ i locMi m Di cxrimi m. 

S\» K Ml, CM IIOKM \ 



Dvsvription of \jjfnc> 

I he I tnplovment l)e\ elopinent Department (M)D) is headquartered in Sacra- 
mento. ( aliloinia. sci\uie the state's population through some 200 field 
otticcs. ill addituMi to Us traditional einplovment service functions. lil)D\ 
mai.M scrMce lesponsibilitics are to pro\ide labor exchange services" for job 
^eekets and emploveis. plus training and employment services for disadvan- 
taged clients. I 1)1) has been in operation smce has a current staff 
between l.^.ouiand 1\()(H) and an operating budget; of $.^84 million, with most 
tunding pioMded Horn I ederal sources. As a labor exchange. liDl) provides 
lob ,eekers and empKneis wuh labor market information, job requirements, 
job-search methods, pl.icement ser\ices and folUm-up. f or disadvantaged pcr- 
Nons. I 1)1) oper.ites a vaiiei\ of client services, including those organized 
through WIN and C 1 lA. and specialized service centers and assjssmem 
programs. 



IK'svripiion of Assvssnu'nl ProKtam 

Ihc vucrall obicctue o] I DD's assessment program is to help individual cli- 
ents tuKl mcaninetui employment through accurate evaluation of their skills 
and potential and tv> match each client with potential employers. .Assessment is 
designed on an uuli\idual basis and grows out of the one-to-one interaction 
between Jieni and [ 1)1) inier\iewer and counselor. Information is gathered 
toi appiopiiaie clieni counseling, job matching, or referral to ■•>ceializcd 
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training programs. To accomplish liiis, HDD utilizes a wide range of stan- 
dardized assessment tools, including: 

Basic Oicupanonul iMeruty Test (BOLT}, both linglish and Spanish ver- 
sions (see Vol. 11, Section II). 

General Aptiiude Tesr Battery iUATBh both hnglish and Spanish ver- 
sions (see Vol. 11, Section 1). 

Nonreadm Aptitude Test Battery (SATB), (see Vol. 11, Section 11). 

Specific occupational test batteries. HDD currently has 466 occupational 
tests, keyed to specific occupations and to the DOT system. These are 
used to provide selection services to employers, where employers have re- 
quested test-selected referrals. .Ml have been validated, and many have 
been revalidated to obtain relevant minority group data. 

In addition, ItDD utilizes specially developed application and screening forms 
in order to gather data on client demographics, job histories, actual and poten- 
tial interest and skill areas, and client eligibility for special programs (such as 
C'FTA and WIN). These forms are keyed to Department of Labor require- 
ments for information and are used by intake intcrview^*rs who assess client 
service needs. .Assessment is conducted at various loc^ and regional offices 
throughout the state which are linked by a computer ivetNvork. 

How the Assessment Program Developed 

The assessment program grew out of liDD's function as the state's primary 
emplovment service. While initially a traditional labor exchange and adminis- 
trator of Federal unemployment insurance and disability insurance benefits, 
has contiiHiously upgraded and expanded its assessment program to keep 
pace with l ederal programming (such as CIVFA and WIN), client needs and 
labor market conditions^ 

C urrent Operation of the Program 

The assessment program takes many forms, depending on client skill level, 
empUnment history, employability and training needs. In general, clients are 
first interviewed bv intake persiuincl who determine eligibility for specific pro- 
grams, conduct initial ass,.^menl. and link clients with placement inter- 
viewers, emplovment counsellors, social service or other special program per- 
sonnel vvho conduct additional assessment if needed. Assessment can Kcur in 
the following major program Cinnponents: 

I. T.mplovnwnt Servuw This is the basic labor 0Xi:hange function, where the 
objective is matching iob-ready clients with existing jobs. Following the 
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intake interview, clients not elitfible for special programs but having mar- 
ketable job skills are interviewed; assessed as needed through use of GATB 
or specitK vK-cupational tests; instructed in job-search techniques; or given 
vocational training. oHices throughout the state share job orders and 
labor market data to oHer clients the greatest chance of finding employ- 
ment. 

f or clients who need them. l!Dl) provides specialized services to over- 
come employment barriers such as lack of education or training, health or 
personal problems, lack of transportation or child care, or physical and 
mental handicaps. If necessary, the intake interviewer, placement inter- 
viewer or imnloymei.t counselor refers .-lients to the Department of Public 
Sivial Services. 

. StrvHv C\-nh'r Pruarum. C lients who are disadvantaged and have special 
problems are referred to one of eight Service Centers located in economic- 
ally depressed areas of the state, liach C enter houses an HDD office and 
may include representatives from other agencies who can provide services 
such as legal aid. housing, family service, education and training. Clients 
arc assigned to full-time case counselors who develop individualized plans 
tailored to specific client needs, l-ach client is assessed by the ca.se counselor 
through basic paper-and-pencil tests (such as the CiATB and NATB), and 
each Is counseled and assisted in finding employment. 

t he counselor is responsible for guiding the client through the stages of 
assessment. empUnment orientation, job development, job referral, train- 
Mig and. if needed, referral to outside agencies. The major difference 
between assessing Service Center clients and Kmploynient Service clients is 
MI the amount of personalized attention and services of other .social service 
.igencics available to Service Center clients. 

(.1/1 and WIS. 11)1) ,s mandated by the state to administer CHTA 
balance of state grants to the 28 rural counties not populous enough to be 
prune sponsors on their own. In this capacity. l-DI) works with local agen- 
cies to proMde manpower assessment, training and resource and referral 
seivices Once C M A eligibility is determined by the intake interviewer, 
assessment tolious the general l-DI) l-mployment Service pattern. 

In Its WIN ptogram. I DI) provides clicms with services such as labor 
nuuket exposure, lob-findiiig workshops, individualized job-search plan- 
ning, iraiiiiiig. career opportunity development, public service employment 
and work experience. Assessment in the WIN prvigram is split into two 
parts; regular assessment, which is conducted by I-;i)D using the Hmploy- 
meni Service basic model, and social service needs assessment, conducted 
b\ the Depaitmeni of I'ublic Social Services. 

In c.ich ot Its progiain aieas. l-DI) assessment is conducted by intake 
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interviewers, placement interviewers or employment counselors. Selected 
EDI) timploymeni Service personnel receive special training in selecting, 
administering, iniegraiiiig and utili/ing assessment tools and data. I his 
training is conducted hy l.DD on a regular in-house basis. 

StreiiKthfi of the PruKrtim 

EDDN assessment program has several important strengths: (1) Although it is 
a large agency, assessment is geared to the needs of each client. For clients with 
marketable job skills, little in the way of formal paper-and-pencil assessment 
occurs, lor disadvantaged clients with few job skills, formal assessment cou- 
pled with supportive guidance, counseling and training provide data that coun- 
selors and job developers need to be most effective in helping individual clients 
find jobs. (2) avoids overassessment and concentrates on obtaining basic 
data sufficient to meet each client's specific needs. (3) All EDD assessment is 
directly linked to Department of l abor and other Federal guidelines, ensuring 
comparability with Employment Services in other states and expanding the 
client's markeiabilitv beyond local or regional labor markets. 

Problems and Pitfalls of the Program 

Two specific problems have been identified within the assessment program: 

1. Assessment turn-around time tends to be lengthy, costly and often counier- 
producrive (initial intake is usually swift, but a high volume of clients often 
creates lengthy waits for employment counselors in some EDD offices, with 
the result that clients become discouraged, may not return, or may feel that 
FDD is unable to help at all). 

2. Other agencies, such as local college, high school- or the Veterans Adminis- 
tration, are not fully utilized or linked to the I DD system, thus wasting 
potential resources :\ r specialized assessment. 

Future Plans 

FDD projects sevof \1 future trends lo be developed through or in conjunction 
with the L'.S, Fmployment Service: 

I. A new USHS I merest Checklisi and new USES Interest Inventory are cur- 
rently under development to aid counselors. These are ea.sy-.io-administer- 
and-interpret assessment tools designed to tap client job interests and are 
keyed to the Department of labor's forthcoming Guide for Occupational 
Exploration. These toois, along with the GATB, will shorten turn-around 
time and provide a standardized and validated job preference screening 
system. 

IS 
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2. linkages Nvnh undcrutili/cd local a^'ncics nviII be established to provide 
more in-dc'p:h and comprehensive assessment programming. 

V Sta.dard assessment tools, such as the CiArB. Nvill be administered and 
sewed hv compute, (th.s ,s a long-range goal, at least tiiree to four years 
awa> trom being operatioiuili/ed). 



Pk.h.ksm (■MAlsuin.KuK, Si k\ lu t ui , u,i .„ nu 

Si ul 111 \s. Vk\| , \ i\i lOKM \ 

Description of Aitenc) 

tol'l^" '-^.V"'*;''! 1'"'^' ^'"^"^^ Headquartered in Visalia. a small 
town Uatcd ,n ( alitornia s c entral San Joaquin Valley agricultural region; it 
serves lulare t ounty's population ot over 2()0.(KK). CUIS is an arm of the Col- 
lege of the Sequoias, subcontracting to the local CETA prime sponsor, the 
lu are C ountv Human Services Agency. C^UlS's major objectives are to pro- 
VKU C MA ehgibiUty. assessment and training programs to all eligible man- 
povver clients vv.th.n the countv . To attain these objectives. CUIS maintains a 
.oun v-w.de outreach program, determines client eligibility, assesses voca- 
lUMia skills or clients develops d.ent employability plans, identifies appropri- 
ate U- lA and non-C MA referral programs, and assigns clients to programs. 
C L S ,s tunded on a yearly flat-rate contractual basis. Clients include econom- 
ically disadvantaged minority and unempl.-.ed per.sons 18 years of age or over 
who have tew or no job skills. 1, has been in operation for over fwo years and 
processes about 5(HX) clients per year. 

Description of .\s.scssment Program 

VVith Its niaioi obiectives the screening, preparation and orientation of eligible 
c lents tor either training or employment. ClJIS's efforts are directed towards 
uient motivation, assessment and commitment to strive for realistic self- 
generated goals, it does not provide on-the-job training, but refers cliems in 
r.eed ot such services to other subcontractors. Assessment focuses on motivat- 
ing clients toward a self-directed search for employment and is designed to 
assist clients m job-search techniques, clarifying job skills and interests, under- 
standing appropriate employment-related attitudes and values, and r- <ucing 
barriers to emplovment. To accomplish this. ,he assessment program consists 
ot a tive-dav process, centering on a one-and-a-half-dav intensive workshop 
tfiat uses small group interactive techniques. During the workshop, vocational 
counselors guide clien's through a series of career development exercises 
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including employee enipKner awareness training, confidence-building exer- 
cises, and success moiivaiion activities. At one point during the two-day work- 
shop, chenis are administered a single papcr-and-pencil assessment tool, the 
ItusK' CkKupudimul l.i(erui.'\ lest (BOl I) [see Vol, IK Section IIJ, 

Mie BOl I is i»>ed to obtain reading and matli scores after preparatory dis- 
cussions about tests and test-taking and pretests. Hie client's performance on 
the preie>is deternunes wliicli level of BOl 1 will be administered, lo avoid 
built-in failure and thus client discouragement. 

After the workslu^p ends, each client receives individualized counseling 
from the instructor counselor about his licr progress, goal . what is currently 
available througli t'l- lA prv^grannning or other services anu what classroom 
training is available. If needed, tlie client is referred fo^ i-u ning in a number of 
possible areas including: (a) work experience, (b) nontrauuional women's 
careers, (c) lob orientation, (d) work readiness and job-keeping, and (e) special 
careers tor the liandicapped. In eacli of these, a small-group counseling format 
is used. I or client^ who need OJ I. referrals are made to an appropriate sub- 
contractor. 

HoH the .\ssessnient Pruitram l)e\eloped 

Some three vears ago. the C'l: \.\ prime sponsor approached the College of the 
Sequoias about establislung a centralized assessment program. Funding was 
secured bv contract from the C'li r.\ prime sponsor, liach year, the College is 
required to compete for Cl IS funding. At first. CUIS used the Micro- 
roVVl-R work sample system (see Vol, 11, Section 11), As client volume 
expanded, however, tliis was replaced with the current workshop system 
because: (1) Cl'lS was dealing witli an increasing volume of clients. (2) turn- 
around time was critical for the program's success, and (3) the local labor mar- 
ket was not highlv technical and typically did not need the kinds of skills a 
work sample svstem was capable of assessing. Tlie workshop system addressed 
these problems and was able to capture and capitalize on subjective variables 
(self-ciMifidence. motivation) vvlucli enable clients toassess and motivate them- 
selvcN. Client evaluations of tlie system have been positive. 

Currvnt Operation of the Program 

C lieiiis ciMUe to C I IS from tlie state I tiiploynient Service, subcontractors, 
and through self-referrals. C I'lS advertises in the local media to attract clients 
and publicize the program. I he typical '1-mployment Motivation A,ssessmeni 
Process** is as follows: 

1. FliMihilitv I vuluurton Phu^c. Clients* eligibility for the program is deter- 
mined bv intake cligibilitv interviewers. In addition to collecting demo- 
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graphic and socioeconomic data, the eligibility interviewer gathers infer- 
mat.on about work history, skills trainitig. special talents and career aspira- 
nons. I hese data are recorded and transferred to the client's counselor for 
later use. Ineligible clients are referred to other service agencies. Those cli- 
cuts with immediately marketable job skills receive counseling and are then 
re erred to an appropriate subcontractor. During intake, clients are given a 
take- home, selt-scoring practice main test and are asked to bring the com- 
pleted test to the workshop. 

2. Employment Klonvatum Assessment Workshop. This one-and-a-half-day 
session IS broken uito three half-day segments. The first half-day is devoted 
to educating clients about CTTA through small-group interactions and 
didactic presentations and administering a math and language pretest. 
Here the objectives are to develop a self-help environment, inform clients 
abou the assessment program, discuss expectations and increase motiva- 
non or employment. During the second half-day. clients are given the 
HOI I (pretest performance determines what level of the test is to be admin- 
istered to a given client). In addition, clients are guided through exercises 
des.gned to identify individual attributes, occupational skills, job barriers 
and goals The third half-day is devoted to helping clients develop an 
unployab.l.tv plan. The counselor-instructor leads clients in specialized 
activities where they learn to identify personal an-1 vocational skills, narrow 
employment interest choices, match career goals with realistic methods of 
achievmg them and evaluate their own commitment to succeed in CETA 
programs and to obtain long-term employme:it. Workshops are conducted 
hv protessional vocational counselors. C'UIS staff members are trained and 
cer if.ed in the administration, scoring, interpretation and use of the BOLT 
nv i .s. Departmeni of I. abor trainers. 

•V Counselor Assessment Integration. After reviewing data from the assess- 
f'eni workshop, the counselor-instructor meets with each client individu- 
a Iv for one hour. Together, they focus on career objectives, realistic goals 
streii}.! hs and weaknesses, motivation and commitment to succeed and 
available employmem programs. Tentative career choices are made, and 
the client is informed that his her decision will be represented at the selec- 
lion commntee meeting, which is the next step in the assessment process. 

4. Seleaion Committee. C ounselors meet twice weeklv to select clients for 
counivwide CT lA programs. I-ach clienfs employabilitv plan is reviewed. 
an.1 clients are ass.gned to available openings based on their motivation, 
work rcidiness. economic need, abilitites. work values and cmplovment 
t^arriers. Because there are limited numbers of openings, clients not 
rcteired to C I lA programs may be offered other services; work readiness 
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training, job search oriciuaiion; Clhl) programs, community resource 
referrals, or special programs for clients who are hard to place. 

5. Placement Servwes, when needed, arc provided by the state I'.mploymeni 
Serv icc, ubcontractors or county agencies. 

StreitKths of the PruKram 

CDlS's I'mploymeni Motivation Assessment Program is unusual in its empha- 
sis on individual client self-motivation and movement away from more tradi- 
tional asscNsmeni techniques. Its specific strengths are these: 

1. Clients participate actively in assessment; evaluate their own occupational 
interests and abilities; and develop goals, an employability plan and a sense 
of commitment to the program. 

2. Counselors piovide a onc-to-onc basts for interaction with clients at each 
stage of the assessment program* use their time more effeciivety, promote 
maxiimini client learning and help clients to help themselves. 

3. Overall, the entire program can result in greater cost effectiveness in the 
long run, higher client success rates, fewer client complaints of unfairness 
and bias, lower counselor **burn-out** rates and effective utilization of 
employment resources. Some data are available from CUIS to support 
these assertions. 



'Problems and Pitfalls of the Assessment Program 

Although C*UIS\ program has been successful in terms of client volume, client 
feedback, CL T.A goals and placement success, some problems have been iden- 
tified: 

1. There is a need for additional staff personnel, vocational counselors and 
instructors to mamtain a high level of personalized service to clients. 

2. There appears to be some co.'.flict in the expectations of CUIS, the CETA 
prime sponsor, subcontractors, clients and employers. 

.3. Centralized assessment of some skill areas, such as typing and clerical, is 
iweJed. 

4. A fragmentation of services often results from the relatively large number 
of subcontractors and the physical distance between all components of the 
system. 

5. Oigani/aiional and budget constraint problems tend to impede overall pro- 
gram effectiveness. 
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Future PUns 

CVIS aniicipaies thai budget, organi/aiional and related problems will be 
resolved as then svsicrn gains experience over time and as new service areas are 
ideniitied for program expansion. The issues ot interagency communications 
and interface with the communitv's employers are currently being addressed 
and are particularlv vital lor tl.'lS's future. 



Pk( M .K \M M wj'i »\s I k Asm ss\u \ ! \ Pi \^ I \u s I s n M (MAPS), 
Pi \^ \ ww.Cw iM)ksi \ 



Description of At(enc> 

MAPS is headquartered in Placentia, California and serves the high school 
student population of several school districts. As an organizing system, 
MAILS' primary objective is to offer counseling, a.ssessmenf and placement 
services to high school students who need employment. It includes economic- 
ally disadvantaged, minority and handicapped students as well as those from 
economically advantaged backgrounds. MAPS has been in operation since 
September 1978, and approximately 2,600 students have enrolled for services 
since thai date. Funding for the .^rogram comes from both Federal and state 
sources. 



Desiription of the .\.vsessment ProKram 

I he goal of M.APS' assessment program is to obtain a level of data on client 
inieresi and skill levels sufficient for counseling and placement services or 
referral to training programs. Assessment is designed to gather only enough 
information for these purposes, and MAPS attempts to avoid over-assess- 
mem. The following assessment toolsare used: 

(OAIS lob-niaichini!, cmployability attitudes and skills components (see 
Vol IK .Section II). 

1 1/ PAfi svsiem (sec Vol. IL Section II), 

( all forma (krupufian Preference Sysiem fCOPS) and (ureer Ability Place- 
ment Survev /( APS/, both commercially available job interest and aptitude 
screening devices. 

Piaoruil Interest I xploration Survey fPItS). a commercially available job 
inter csi screening device (see Vol. IK Section II). 
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Assessment is conducted at the local high schools. MAPS has a large, self- 
contained camper which is used to transfer the work sample equipment and is 
used as a mobile assessment center. 



How the Assies.snieitt ProKram Developed 

The MAPS program was created in response to the need for reducing youth 
unemployment in the area it serves. Funded by CETA, the prime sponsor is the 
North Orange County Regional Occupational Program. 

Current Operation of the ProKram 

MAPS is operated through the Placentia high school district Each high scJiool 
in the service area has a career center staffed by counselors and each has a 
MAPS computer terminal for client data input and job-matching output. The 
MAPS assessment process consists of the following steps: 

L The MAPS program is publicized at each school through posters, descrip- 
tive literature and personal contacts with counselors, teachers and students. 

2. The student completes an application form. CETA eligibility is determined, 
but non-CETA eligihles are also accepted. 

3. The COI^S instrument is administered to all students. About half of the stu- 
dents complete the COPS instrument, depending upon the counselor's 
determination of need for the data it provides. For students who are educa- 
tionally, physically or emotionally handicapped, PIES is used. COATS 
and VALPAR are used as needed. At each school, counselors have the tlexi- 
hility to add other standardized assessment tools to the battery if they 
believe that more information is needed. 

4. After formal assessment is completed, data are analyzed and used by the 
counselor to select an appropriate DOT-keyed employment area. Both the 
student arrJ counselor work together to make this selection. The student is 
informed of the assessment results, with the counselor interpreting their 
meaning in terms of strengths, interests and weaknesses. Short- and long- 
range career goals are established from this interaction. 

5. Assessment data are entered on the MAPS computer through terminals at 
eUch school. The computer is then used to match the student with specific 
DOT job categories and existing employment services. If CETA-eligible, 
the student is referred to a CETA agency. If not CETA-eligible, a non- 
CI* TA agencv or specific employer is selected for referral. Assessment data 
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arc made available lo ihe referral agency. All MAPS referral agencies have 
computer terminals; hard copy is provided if requested, 

6, If the student is unable to secure employment or training, the counselor 
reviews available data, confers with the student, and another referral 
source and / or employmeni area is selected. 

Assessment is conducted by a variety of personnel at the various schools, 
including academic counselors and career guidance counselors. At each 
school, individual or group testing is used, depending upon the client volume. 
Training for assessors varies from school to school except in the case of pro- 
fessional guidance counselors whose education specialty equips them with 
assessment-related skills. 

StreiiKths of the Program 

Major strengths of the MAPS program include: (I) a direct in-school service is 
made available to a large number of potential clients prior to graduation or 
voluntary termination of high school (this enables MAPS to reach students 
before they become unemployed or underemployed); (2) the use of an on-line 
computer to input, retrieve and analy/e assessment data and then link it with 
DOT job categories reduces turn-around time; (3) assessment results are made 
available to referral agencies through the MAPS computer system, thus help- 
ing agencies to sei vc students efficiently. 

Problems and Pitfalls of the Assessment Program 

Two major problems have been identified: 

1 . Iraming for counselors is needed, particularly in the effective use of assess- 
ment tools such as COArs and VAI.PAR and in the integration of data 
generated from the assessment process. 

2. I uikagcs wuh other employment and training agencies (other than in- 
l4>rmal contacts and cmifercnces sponsored by state or Federal agencies) 
aic needed lo fill the professional and technical vacuum that MAPS often 
tinds Itself in 

Future Plans 

M \PS anticipates an increased emphasis on counselor training, provided that 
adequate funding and staff time can be secured. Recruiting, hiring and train- 
mg counselors to augment existing staff is a particularly important priority for 
the future. 
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An xHHM.C' m UORNIX 



Description of At(«^nc> 

The Assessment and Recruirnicnt C enter is located in Anaheim, California, 
and serves a large catchment area in northern Orange County. Clients come 
from lower socioeconomic groups; about two-thirds are seeking employment, 
and one-third desire job training. The Center receives financial support largely 
through contracts with three North Orange County employment and training 
agencies. The t outer began its full-scale assessment programs in May 1978. 
The Center's contracts require processing a total of 3,6(X) cuent.s and assessing 
at least 7(X) clients per year. 

U«.scription of Assessment Program 

The goal of the Center's assessment program is to determine the client's cur- 
rent job-related skill level, employability potential and job interests. To 
accomplish this, the following are utilized: 

r0.4r.S job-matching, living skills and employability components (see Vol. 
11, Section II). 

yAl PAR system (see Vol. II. Section II). 

Stronii Camphell Interest Inventory, a widely used, traditional occupational 
interest profile (see Vol. II, Section I). 

Career Assessment Inventory, a paper-and-pencil assessment tool designed 
for clients with some post-secondary education who are job-ready. Scores 
are provided on three basic scales: general occupational themes, basic inter- 
ests and specific occupational interests. 

Temperament & Values Inventory, a paper-and-pencil inventory that 
assesses clients' temperament and values for comparison with average 
norms. Results are used to assist in matching a job-ready client with ap- 
propriate occupational categories. 

Oranae County Manpower Commission H'ork History Farm, an in-house 
device that gathers data on client special skills, previous employment, spe- 
cial training and interests. 

Assessment Summary Form, an in-house tool used to give client.s and refer- 
ral agencies a brief summary of a.ssessnient data. 
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Client OhservatioN f orm, an m-house form used to record observed client 
bchaMor during a>>e>smcnt. The referring counselor, assessment technician 
aiKl vocational counselor li>t coninient> on aieas such as ability to communi- 
cate-, assert iveness. appearance and physical limitations. 

hmplovnwnt & Inunin^ Plan f orm, an in-house tool used bv counselors to 
summarize the client's goals, interest ., abilities, barriers to employment and 
training and recommendations. 

Assessment is conducted at the C enter's assessment facilitv-a recently 
acquired, modern, single-storv building that houses staff offices, client coun- 
seling areas, work sample stations and COATS audiovisual systems. The 
C enter is headed bv a lull-time coordinator, and its staff includes .specialists in 
assessment, recruitment services and vocational programming. 



HoH the .\s.ses.smeni ProKram Developed 

The Center is an arm of the North Orange County Conmninity College Dis- 
trict. It was developed to supply assessment and recruitment services for three 
major North Orange C ounty employment and training agencies. 

C urreni Operation of the Program 

Cliems are referred to the C enter from other agencies in the catchment area, 
rhey spend from 2 ' : to I S hours in the as.se.ssment proce.ss. Once recruited and 
referred to the C enter, the typical process for clients is as follows: 

I. An initial interview with a counselor determines eligibility and currem level 
of job readiness. Vox clients who are appropriately skilled, referral is made 
to either the state employment service or other employment programs, such 
as Public Service l-'mplovment (CM TA). 

:. Most clients are given the COATS job-matching module. The COATS liv- 
mg-skills nuHlule and the VAl.PAR work sample and evaluation system are 
utih/ed with clients for whom they are appropriate. Oiteria for use of these 
assessment tools include the client's educational level, career interests, 
work history and phvsiatl or emotional problems. 

3. f or clients with higher levels of formal education (college and trade 
schools), specialized skills, or eligibility for Public Service Employment 
programs, the Strong-Campbell Interest Inventory. Career A.sses.sment 
Inventorv. and Femperamem & Values Inventory arc administered. The.se 
instruments are used bv vocational counselors for assisting clients in transi- 
tion to other careers. Work samples are not used for these clients. 
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4. Results from assossnient arc interpreted by a counselor who works with the 
client to arrive at a comprehensive plan that will lead to additional training 
or employment. 

5. All ( l- TA-eligible clients are referred to an appropriate C"I:TA agency or 
program. C lients not eligible tor CliTA programming receive placement 
recommendations and are referred to local employment services. 

l-ornial assessment of clients is conducted by vocational counselors and assess- 
ment technicians. The counselors are college-trained professionals with spe- 
cialized assessment training. Technicians are paraprofessionals w ho are trained 
by the Center in a thret'-month. semiformal training process. As.sessment 
results are interpreted to clients only by professional counselors. 

StrenKths of the ProKram 

The Center's assessment program has several important strengths: (1) an abil- 
ity to handle clients with widely diverse employment histories, educational 
achievements, career interests or special physical or emotional handicaps; (2) a 
phy.sical plant designed to accommodate a relatively large client volume and to 
facilitate the use of work sample systems; (3) a well-trained professional and 
paraprotessional staff that is skilled in the use of the assessment tools; (4) a 
commitment to effectively utili/e high-technology audiovisual and work 
sample systems. 

Problems and Pitfalls of ihe Assessment Program 

The major problems identified by the Center are: ( I ) a time lag between as.sess- 
ment and utilization of results; (2) overassessment of some clients who, 
baause of existing skills and abilities, do not need full assessment services (for 
example, clients who are functionally literate do not need to be scheduled for 
the Living Skills component); (3) inability to assess clients with .severely limited 
tnglish skills (such clients need to be referred to other agencies where basic 
language skills can be provided); (4) a tendency for job matching to cluster dis- 
advantaged clients into low -need jobs; (5) a need for more trained counselors 
to interpret and utilize the resuus of complex and subtle a.ssessment data. 

Future Plans 

To resolve some of the problems identified above, the Center has taken actions 
to reduce turn-around time to not more than eight days and hopes later to 
reduce that to five days. The Center is adding more COATS hardware and 
increasing its facility size, is attempting to reorient the assessment process to 
minimize overassessment and is referring non-English-speaking clients to a 
local mini-school prior to full-scale a.ssessment. 
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the ( enter also plans to add the JliVS system to its assessment process and 
to add a Msual-acuity instrument. It is important. Center staff have discov- 
ered to measure visual acuity of clients accurately. I inally. there is an cxpecta- 
lon that contracts uiil be added to assess an additional «X) to 70() clients in 
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n«.scriptiun of A^enc) 

Jll^'%l^.llT Community Institute, located in Edmonton, Alberta, Canada 
pop 5(X).()00). presently provides training in offset printing and office skills 
or clients over 17 years of age w ho have not been satisfactorily employed due 
o personal social and 'or economic difficulties or insufficient formal educa- 
tion, l-nrollment in each program is limited to five or six trainees at a time and 
lasts I to .0 weeks. Adjustment to the work environment is an integral part of 
the skill training and is supported by .sessions in human relations, interviewing 
skills and temporary work experience placement. The Institute draws two 
thirds Of Its clients from native Indian populations. Immediate employment on 
completion ot training is the major goal of the program. Funding comes from 
t ederal and provincial manpower agencies and from private grants. 

l)e.M.Tipilon of Assessment ProKram 

Applicants for the Institute's program are asses.sed to ascertain readiness for 
training, aptitudes, motivation to work and need for training in offset printing 
or office skills. Because of special p.oblems in assessing native Indians and a 
low client volume, the Institute does not utilize standard paper-and-pencil 
assessment techniques. Instead, each client completes a brief application form 
and IS then interviewed by each of three Admissions r ommittee members; one 
part of the interview process is geared toward assessing vocational aptitude. 

Ho» the Assessment Pro|{ram Developed 

In designing the assessment component, three factors were taken into account' 
(1) Referring agencies were doing .some screening, thus eliminating a need for 
an elaborate intake process; (2) most applicants were native Indians-inter- 
views w,th them would be of a cross-cultural nature and designed to overcome 
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reluctance to revealing personal inforinaiion— while other applicants had a 
history of emotional problems; and (}) the inier\iewers had no professional 
training in interviewing or counseling. With the help of the Institute's research 
associate (from the I niversnv of Alherta), a ♦''^ur-page, easy-to-read admis- 
sions form was created to elicit basic information. The interview format was 
purpt|.sel> left relativelv uiiNtructured on the premise that starting with a car- 
ing, eommon^sensc approach was a better alternative to a crash course in 
counseling or administration of potentially threatening, complex assessment 
tools. As there were no successful tross-cultural assessment models to follow, 
the best guide appeared to be the insights and experiences gained by two inter- 
viewers who had worked with natne Indian men and women on other projects 
in nonprofessional capacities. 

( urrent Operation of the Program 

Two hundred and ten individuals (117 Indian) referred by some 30 agencies 
have beeii assessed m two years, with about one^lPt^rd accepted. Assessment 
begins with the first telephone contact made by agency or client. The first 
interview process, lasting about an hour, concentrates on giving facts about 
the program and gaining nWornuuion on the applicant^ education, goals, per- 
sonal situation and emplovment history (if any). The following interview- 
allows for a different interviewer's perspective, as well as the chance to focus 
on potential problems, such as with children or alcohol. The final interview- 
fills in an> information not gathered by the first two. 

l-ach applicant also receives appropriate tests of aptitude in working with 
printing machinerv or for level of Hnglish and knowledge of the typewriter 
kevboard. lurther relevant information may be requested frotu the referring 
agencv. 

The Admissions Committee riiu's each applicant as ''acceptable,** *'not 
acceptable^ or ''reconsider at a later date,'* and the de»:i>ion is relayed to both 
the agencv and the client. If "not acceptable** for the next course opening, 
alternatives are recommended— vocational upgrading, life skills training, etc. 
Assessment results for "acceptable** applicants are used to indicate special 
areas for concentration during the training. 
.Mthough there is no formal training program for interviewers, several crite- 
h 'vc been established for selecting them: (I) four years of experience in 
working with a u.;::^'ni! staff at a local native printing and secretarial firm 
and or equt^^^'ent experience on vrh-'r similar projects; (2) an understanding 
of the native Indian people through a wide range ^.m' p**'sonal friends and con- 
tacts; and (3) a personal ability to communicate effectivelv with nui:' ** Indian 
people. 1 he emphasis, then, is on personal skills in dealing with a unique client 
population, I ack of such skills can result in alienating clients and limiting the 
amount of information that can be obtained. 

M) 



ERIC 



Strenjith-* «f the ProKrvm 

So tat. a tornial Mud> o\ ihc assLNMnont coniponciit of the Institute's program 
has not been made JKnveve. . feedback from referrmg agencies has been grati- 
fving. Panicularlv encouraging have been favorable cotmnents from two 
respected Indian-staffed agencies- Native Outreach (employment placement 
agencv) and Name (. ounscling Services- and the increasing number of refer- 
rals tiom former native trainees. 

The Inst mile's assessment program is especially interesting because of its 
reliance on individual interviews to obtain information from client.s whose 
hackgrtnmd is of a nonmainsiream culture. Specific strengths include: (1) a 
one-to-one interaction between clients and interviewers designed to elicit per- 
sonal inf\)rniatu)n in a friendiv . relaxed, nonthieatening manner; (2) the use of 
mterviews raiher than traditional paper-and-pencil or high technology assess- 
nieni tccliniques. which fosters client involvement in each step of the assess- 
ment process; and ( M lecruitmg assessmem interviewers on the basis of experi- 
ence with Ihc client population rather than on specific formal professional 
tiaining. 

Problems and Pitfalls of the Assessment Program 

[tie time spent oi) interviewing presents a problem for the small staff of the 
Hiedin Institute, t^ut shortening the interviews would be detrimental to the 
overall assessment, -\pplicants for the courses are frequently quite resistant to 
disclosing inforniaiion at^out themselves, however relevant to their admission 
to the Institute. In the Indian people's culture, questioning— especially on per- 
sonal matters — is cvMisidcred i ude and it is deemed appropriate to answer eva- 
sivdv Manv Indians present a kind of passive defensive shield to hide lack of 
confidence in ttie presence of white people, which is often misunderstood as 
lack of nuMivatuMi. Iliere aKo are misunderstandings in communication due 
iv» an inadequate kiu)vv|edge of the l-nglish language. Applicants warm, how- 
ever, lo .m interviewer wtu) is not in a hurry, who questions obliquely rather 
ih.iii directlv. and wlio listens instead of interrupting when conversation drifts 
avvav fu>ni the main lopics of tlie interview. Details of past employment or 
meiuion o\ personal woines. or even of future plans uhich do not appear to 
inchide peimaneiit enipUnnient. are then often forthcoming. 

\pplic.ints who are ium jiidian may present a different kind of challenge for 
the iiiioivu-wei. Iliese clients are frequenilv on the defensive. They feel 
asli.imed to tv on nocmI assistance or tliev fear dl^c^Mnlnalion because they 
tiave fiad a mental illness 
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Future P\m% 



Plans are under way to offer addilionuj courses in t>iher occupational skills 
leading to employment. As yet. it is not possible to foresee the changes which 
may have to be made in the assessment' process described above. 



Analysis of Exemplary Programs 

Many valuable approaches to client assessment in employment and training 
settings are represented by the program descriptions reported here and also by 
the case studies conducted by A, I.. Nellum and Associates. It is the intent of 
this section to abstract a few of the most important features for the rcader^s 
review. 

First, the programs tend to be fairly complex and multifaceted in nature. 
They use more than one technique, and the assessment component is tailored 
to some extent to the individual client's needs, background and capabilities. 
The idea o\ cus(om iaUoring assessment to fit each client as a unique individual 
is an important one, and one that developers of new assessment programs 
should explore carefully. 

Second, these programs tend to have relatively well-trained staff to adminis- 
ter, score and interpret assessment tools. This greatly increases the chances 
that assessment will provide meaningful results/since assessment tools, no 
matter how good, will yield mea.iingless data if not handled by staff with 
appropriate skills. 

Finally, these programs frequently take advantage of advances in technol- 
ogy, such as computer processing for information analysis, storage and 
retrieval components of the assessment system. 

The role of the state Employment Service varies greatly in the American pro- 
grams reviewed here. In some cases, the Employment Service is an important 
influence and may render assessment se. vices; in other settings, the contribu- 
tion EScan make is undervalued or ignored altogether. Also, it is important to 
note that these programs tend to conduct assessment in-house, rather than 
using an outside subcontractor. 

Major Problem Areas 

Much progress, the preceding descriptions tell us, is being made in developing 
assessment programs thai can contribute meaningfully to improved service 
delivery in employment and training agencies. Both research evidence and 
common sense are being used in the development and refinement of such pro- 
grams. However, n is equally evident that a number of operating problems 
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remain. The problems outlined below are based on the five assessment pro- 
gram descriptions, the two reports (one with its own set of case studies) 
described earlier in this chapter, and on the author's own informal observa- 
tions \ia contact with hundreds of etnploytuent and training agency profes- 
sional staff over the last several vears. 

1. AsscssiiKMH in omplovmeni and training agencv settings scenis to be in a 
considerable Mate ol fhi\ at the present time. Manv programs are new. 
and in other cases, internal cluiiiges seem to be the rule rather than the 
exception. It is anticipated that this fluctuation will increase over the next 
vear or so. as the impact of the new CMA reauthorization (with its 
specific encouragement ot client assessment) is felt. 

2. l-ven in some of the exemplary programs reviewed in our case studies, 
instruments are being used that appear inapplicable for a substantial pro- 
portion of the target population for these employment and training agen- 
cies. These are tests designed for use with the mainstream, nondisadvan- 
t aged population having no limiting aversive reactions to tests or limited 
experience with the kinds of behaviors testing procedures require. This i- 
surprising, in light of these agencies' otherwise strict attention to the indi- 
vidual client needs where client assessment is concerned. 

V Assessment techniques designed to measure psyehopathology and that 
require a high degree of sophistication to administer, score and interpret 
are being used. Tests such as the MMPl and Rorschach have no place in an 
employment and training setting except under the most specialized of con- 
ditions (e.g.. an agency program directed at a significant number of 
recently released mental patients, with careful participation in and super- 
vision of testing by a clinical psychologist or comparably trained individ- 
ual). l-:ven generally good training in assessment does not provide an ade- 
quate base for use of the^-e very specialized tests. Their lack of relevance to 
most employment and training goals is sufficient justification in and of 
itself for their eliminatio'i. 

4. .Assessment staff are often inadequately trained to administer, score and 
interpret results from techniques their program uses, even if the devices 
themselves are appropriate. 

5. There is a tendency toward faddism in the use of assessment devices 
brought on m part by the increased appearance ot devices offered by com- 
mercial firms. These firms engage in substantial marketing efforts to con- 
vince emplovment and training agencies to buy their products, and in 
some cases, the intensity of the marketing effort may obscure the lack of 
worth or lack of relevance of the particular assessment tool. 
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6. Although itucgraiion ot assessment into the ser%ice delivery function is 
perhaps aeeonipliihed better tor the case study programs reported here 
than in most other situations, there is still a problem of physical distance 
and lack of communication that inhibits full utilization of assessment data 
in the dec'sion-making process. 

7. Although techniques are needed now, loday, many of the promising 
assessment techniques discussed in this volume are still in a fairly early 
stage of dc\elopineni. lAen those created by the U.S. [employment Service 
and subjecu-d to extensive field testing and evaluation are still relatively, 
recent, leading to an inescapable conclusion: Where severely disadvan- 
taged populations arc concerned, the resources for assessment are definite- 
ly underdeveloped. The situation is getting better, but the problem of 
shiirttall remains serious. 

8. Using a stringent rule of cost effectiveness, some assessment procedures 
collect uto much mjormatum relevant to given goals. A simple checklist of 
job history may suff ice it only three possible jobs are open. When the goal 
is to place many applicants as quickly as possible, fine discriminations 
based on sophisticated measurements can be both wasteful and useless. On 
the other hand, in attempting to identify what factors are as.sociated with 
long term program success, complex assessment of the client and his/her 
environment may be \\arranted. 

"i. Methods for evaluating the utility of assessment programs in local employ- 
ment and training agencies are underdeveloped; many programs are not 
evaluated at all. 

10. Some counselors — and clients— resist assessment programs because of a 
general suspicion of psychological assessment, unfortunate prior experi- 
ences and a variety of otner problems. 

11. There is now legal liabilUy for all employment and training agencies with 
respect to assessment, bilsed on the Uniform Guidelines mentioned earlier: 
actual lawsuits are being t'iled by clients claiming that assessment discrimi- 
nated against their, in being placed for training or employment. 

12. Often there are unrealistic expectations about what assessment can yield, 
and this. Ill some cases, has been fostered by commercial developers 
promising much more than they can actually deliver. 

13. .Assessment sometimes is used as a crutch or as a way of distanc^ing agency 
staff from clients. 

The effective client assessment program must grapple with these problems; 
some suggested problem-solving strategies are given in Section HI. 
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III. DKVELOPING AN ASSESSMENT PROGRAM 



WhatVs Covered in this Section 

The model for client assessment programs presented in the first part of this 
section outlines the basic functions of assessment in a deliberately simplified, 
linear fashion. It is intended to provide a foundation for program developers 
to adapt and elaborate on. depending upon what combination of purposes 
assessment serves in a given agency setting (see discussion in Section I on pur- 
poses of assessment). 

Further conceptual underpinning is provided by Bruno's (1978) approach to 
ciassitying assessment programs, summarizvd in the second part of this sec- 
tion. Program developers can use both these sets of concepts about assessment 
when addressing the early steps in this section's planning checklist. All of this 
material constitutes input to the creation of a process or structure into which 
the content of assessment— the techniques described in Volume Il—will fit. 
Developing such a structure is just as important to success in cliem assessment 
as selecting (or creating) the best possible assessment tools to serve given pur- 
poses in an employment and training agency setting. 

t 

A.ssumptions 

The concepts discussed in this section all rest on the same basic assumptions: 

I. Client assessment is not only encouraged strongly for all CETA-sponsored 
programs under the new legislation-it will definition take place wher- 
ever there is more than one service delivery option for an agency's clients. 
That is. some gathering of information from which decisions can be made 
IS inevitable when there are choices regarding which of several options for 
service a client should be directed to. unless one simply flips a coin to make 
the choice. Assessment may be unsystematic, informal, based on counselor 
hunches or what clients ask for. but it will occur. The only question is 
whether a relatively more formal and systematic process will help to serve 
clients better. 

Assessment projirams must be tailored to fit local circumstances: No one 
model or approach, and no one assessment technology, will meet all agen- 
cies' needs. Thus, the planning tools laid imt in this chapter must be evalu- 
ated in light of how a given agency operates, the kinds of cliems it serves 
and the types of services it provides. Also relevant are the background.s, 
capabilities and attitudes of staff toward assessment. 
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.V One of the most basic decisions thai has lo be made about any assessment 
program is whether it will be conducted in-house or subcontracted. Opera- 
tion of a subcontracted program will be substantially different from that 
for one developed or refined in-house. This section will attempt to address 
both alternatives. 

4. Installing or revising a client assessment program is like carrying out any 
other type of significant organizational change— it is likely to work better if 
it IS well-planned, if staff of the organization are decision-involved in the 
creation and operation of the program as much as is feasible, and if there is 
systematic follow-through on program design so that operating problems 
and needs for improvement can be tended to systematically rather than 
haphazardly or not at all. 



Basic tJements of an Assess:. .ent Program 

The following chart summarizes (he most important components of assess- 
ment programs in local employment and training settings. It is intended to be 
tairly comprehensive, including subdiv.sions that may or may not be present in 
a given program. For example, in a relative ly simple assessment effort, setting 
individualized goals for assessment may not be necessary, since everyone 
receives largely the same treatment. Pretesting orientation may or may not be 
pan of the assessment program; depending upon the type of client served, it 
may or may ni>t be necessary. 
l-.Iements in this chart are discussed separately below. 

BASK I ll.Ml-.Ms OK AN ASSKSSMI.N f PK0C;KAM 



s<lrv(ion into assessmenl proRram 



[::z 

[ PmrscinKorifnution 

I — 



Nrttinx individuali/rd Rtmls of nMs^ssmeni 

X 



AdmtnUCnition of assessmrnt techniques 
■ 



Dr bhef inK of vJieni folioHinx completion of asscssmenl 

X 



Si'ohnK of technique administered 1 



Prepamtion of as.srssment results / report 



X 



I* eedback to counselor or other a^enc) staff 

~ \ " — 



Feedback to client 

1 
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KollimMip on »5iJte!(!imriil 

— - 



— 



i: 



Kvaiuaiiun of iiwtesNmcni proKnim 

I — 



HfOKram iinpr<>*cmcn> bimcd un onluaiiun rcNulis 



Sekclion into assessment proKram. For any program serving a reasonably 
broad mix ot clients, it is likely chat not all clients will need to go through the 
assosment program. For example, if the program is geared primarily toward 
providing specialized services for severely disadvantaged clients unfamiliar 
with standardized testing materials, a relatively well-educated and job-ready 
client may not be able to benefit much from assessment. A brief interview fol- 
lowed by a training or placement referral may be sufficient. Therefore, with a 
diverse clientele, it is essential lo be systematic in deciding who is to participate 
in the assessment program. 

SetlinR individualiiced Koals of asses.siiient. Once admitted to assessment, each 
client should be treated as an individual. Within the limits of nexibility inher- 
ent in the nature of assessment services provided, each individual needs to be 
given services relevant to his or her particular needs. 

C lient involvement in goal setting is important. Cliems often are well aware 
ot what information about their vocational needs, interests and abilities may 
be relevant to their employment goals. Many have been through service pro- 
grams before: time and ener-y can be saved by learning what the client knows 
betore proceeding further. Second, client acceptance of the as.sessment process 
IS likely to be higher it goals are set conjointly with the coun.selor or as.sessment 
coordinator. The client will then know what the goals are. can appreciate 
assessment as something that leads to a real end product (i.e.. getting training 
or getting a job) and can begin to develop a sen.se of ownership in the service 
process itself. 

PretesUfiK orientation. Particularly for severely disadvantaged 'clients, getting- 
acquuinted experiences with assessment are critical. Pretesting orientation 
helps to reduce anxiety, increase understanding of how a.s.sessment will pro- 
ceed and help-, further to establish the relevance of assessment to later employ- 
ment and training servues. h, cases where client anxiety is very severe pretest- 
mg orientation could involve the use of .stre.ss-management techniques such as 
relaxation exercises. 

Ihere IS an expression in computer sciences: ••garbage in. garbage out" 
That expression ma> unfortunately apply quite well to an assessment program 
not having pretesting orientation, since it is difficult to interpret as.se.ssmem 
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results for a client who was highly anxious, hostile, or simply confused about 
the assesstneni process. 

Administrtition of asMrssmeni techniques. Though details, of administration 
vary according to the particular techniques^ used, the general environmenial 
context of assessment that is needed for suct?ess docs not. The physical space 
where assessment occurs should he clean, attractive, quiet and relatively free 
of distractions and appropriate both to the nature of the assessment process 
and the kinds of people who are being assessed. Proper test admini.straiion 
also is a necessity. 

Debriefinit of client followinK completion of assessnient* This critical step is 
often ignored m assessment programs that tend to follow the so-called medical 
model — the counselor or assessment staff person as expert and the client as 
passue recipient of service. This approach may create difficulties for at least 
two reasons. First, during debriefing, the client may provide some critical item 
of information that will help in understanding the results of assessment. For 
example, one page of the client's question booklet might have been blank 
because of a printer's error. Second, debriefing helps further to allay client 
anxiet\ about assessment and to provide an opportunity for initial counseling 
about ilie use of assessment results. 

In some cases, wiih gov/d debriefing clients may be able to select for them- 
selves lob options o\ training alternatives. The experience of being assessed, 
espcciallv where work sample approaches are used, can generate many insights 
thai debru'fing \^m\ help turn in:o Ci>ncreie options for taking action. 

ScorinK of each technique administered. It is important to have well-trained 
pcisonnel to score assessmeni devices accurately. Quality control procedures, 
e.g.. periodic double-checks on scoring accuracy, are also highly desirable. 

Preparation of assessment results report. Whenever assessment results are 
piepared lor feedback, the most critical question is, **To whom is this report 
speakuig.^** Assessment results used for administrative or evaluative purposes 
mav laKC quite a difteient form from what is intended for use by a counselor 
or \o\ leedback djrectlv ti>a*.lieni. 

k 

Feedback to counselor or other aKency staff. Whoever is responsible for mak- 
ing decisions about a given client needs to have access to assessment findings. 
It makc^ liiilc sense to feed back oirly an overall rating if there is anything lo 
learn trom a tuller presentatKui of the data about agiven client. 



l-eedb«ck to client. Although clients may need to have edited and simplified 
Ncrsions of assessment results to be able to understand them and to avoid mis- 
interpretation, one should not underestimate the capacity of clients to learn 
rom ^^hat came out of the assessment. For example, giving a client the chance- 
to compare his or her internal expectations about performance with the reality 
can DC a growth -encouraging expeiience. 

Kolhm-up on assessment. It is important to check with the counselor, other 
1*Tm ''''II' " th^' ^-"'^'"t to learn whether the assessmem "filled 

the b,l . I his ,s ditterent from program evaluation, since it is immediate and 
mdividuah/ed to the given client. Such follow-up may be especially helpful if 
tUtf assessment prc^gram is inultifacetcd and few clients are likely to receive he 
total package: w.th tollow-up. it can be determined whether or not any needed 
" '"advertently omitted for a particular 

Records storage. Provision needs to be made for Morage of whatever records 
are required tor tollow-up work with the diem, for program monitoring and 
evaluation purposes and for oil.er administrative purposes. Given current legal 
labilitv for decisions made as a result of assessment, it may be quite imporam 
to keep on hand not onlv any reports prepared but also actual as.sessment 
records such as answer sheets. 

FAaluation of assessment proKtam. C riteria for evaluation of assessmem pro- 
grams were presented in Section I. Having an ongoing monitoring effort is 
important not onlv in order to meet Federal or other sponsor reporting 
requirements but also in relai ion to making the program better over time. 

ProRram improvement based on evaluation re.sults. It should be assumed that 
no assessment program is perfect. The assessment program should be struc- 
tured so that evaluation findings can be implemented and also so that sugges- 
tions regarding program improvement can be submitted by assessment staff 
counselors and other stat, of the agency and by clients themselves. In many 
cases, real improvements in the assessment operation may emerge from the 
suggestions clients make. 

Naturallv. eligibilitv determination and direct client matching also may 
occur 111 (he level .^tvpe programs, and the distinctions between the three 
t>pcs of program models may not always he quite as clear in practice as they 
arc m Hruno s discussion. In terms of the main operating componems of 
assessment programs, however. Bruno's scheme does capture most of the rele- 
vant dmiensions. Referring back to the seven purposes of assessmem given b> 
the author m Section 1. close correspondence can be seen between that set of 
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concvpis and Bruno's "IcvcN ot assessment." except that the latter do not 
include the levels ot program evaluation and participation in research efforts. 

tMearly. the higher a given program moves on Bruno's t>vin hierarchie?? of 
assessment program charactens'ics. the more complicated the program 
becomes. Making this classification initially in developing or refiniiVg an 
assessment effort nia>, therefore, he of value, since it may help program plan- 
ners to appreciate fullv the complexities of Wi.at they are creating. For exam- 
ple, under Progani Model 3. where assessment functions primarily in a track- 
ing or referral capacity, the program should include a mechanism for coor- 
dination of goals and priorities lor assessment among the several agencies, so 
that conflict and unproductive overlap in assessment efforts will not occur. 



Planning C hecklist 

A key obstacle to success in planning organizational changes often is a lack of 
conceniual foresight about what may be the anticipated consequences of given 
courses of action. For example, if h commercially available assessment pro- 
gram is adopted for use without carefully determining what will be its cost, the 
agency may make a commitment to an assessment function that is infeasible to 
support on an ongoing basis. Similarly, inefficiency may result if an in-house 
assessment program is set up without considering carefully whether some or all 
of the assessment functions could more effectively be contracted out to other 
agencies such as the state Hmployment Service. 

The planning checklist is only a starting point in building conceptual fore- 
sight regarding assessment program development. The writer makes no claim 
to have included all of the possible questions one might need to ask when set- 
ting up an assessment function in an employment and training agency. Plans, 
some of which would have to be committed to writing for budgeting or pro- 
posal submission purposes, actually may need to be much more detailed than 
what is represented bv the cliecklist. However, it is felt that most of the major 
elements required for effective program development have been included here. 

The checklist also can be used to review an existing program in order to 
determine what might be missmg, what could be improved and how to respond 
more creatively and effectivelv to tlie challenges presented at the beginning of 
this volume. 

Two mam paths are provided for in the planning checklist. One is for an in- 
house assessment program to be run by employment and training agency staff 
themselves, and the other is for a subcontracted program that would actually 
be operated by staff in another agency, such as a state Employment Service or 
private, profit -making firm specializing in assessment. 
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FolUming the checklist is a discussion ot most of its tuaiir steps. In some 
cases, reference is made to other sections of this Manual to provide the 
necessary explanatimi. 



Pl.ANMM, ( HK.t kl.lSI K)R I)K\ FI.()PIN<. AN ASSKSSMKN I PRmiRAM 

Selling obicciixos ihai relaic lo ihe i:>sency and iis mission 

Making decision as lo locaiion ot asscssmeni program: m-housc vs. subcon- 
traded 

In-houst t^ogram 

Deicrmimng Hhai resources arc available lo suppori ihe assessmeni program .... 
Asceriaming hou asscssmeni v^oiild fri in wiih overall agencv uperaiion- 

including relaiionships uiih UKal emploversand uiih oiher service agencies ... . ' 
C'reaiing planning commiiiee for asscssmeni program uiihin agency. 



Dcicnnimiig kcv jvrson or pcisomn lo be responsible tor asscssmeni program 
dexclopmciii . . . , . . 

Deicrminiiig alicrnaiivcs tor siaft invoKcrneni in program developmeni 

Intormaiion search tor usable assessment techniques 

Reviewing ideniificd lechmqucs 

Sclecimg expcri corisulianis on asscssmeni 

Selcciing packaged icchniqucs or programs 

Developing a rechnique or program 

Adapnng an asscssmeni icchnique or program 

Deicrmimng compliance u iih ihc L nijorm Guidelines on Employee Seleaton . 

Deicrmining ihe cosis ot asscssmeni (maieriais. siaff iraining. faciliiies and asso- 
ciaicd indireci costs). 

J'ormah/ing an overall plan for ihc assessment program 

Setecnng asscssmeni siatt 

Iraimng siafl 

Hiring siatt or consulianis tor desired validanon cfforis 

Regulan/ing ongoing oivraiion and iroublc-shooiing 

Implcmeming evaluation and program improvenieni etforis 

Prcseniing assessriicni programs m funding proposals 
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Subcontraaed Program ^ 

Defining needs tor assessmeni inpui i : 

Considering subconiracloraliernaiivcs ; ; 

Siaie limploymeni Service? 
Universiiy or c'oUege? 
Private nonprof it? 
Private tor profit? 

Making contav.t wiiti potential subcontractors 

livaluaiing thealtertiative 1 ; 

Detertnining costs of assessment (including indirect costN not in cstitnatcs of 
assesstnent providers) ; * 

Writing the subcontract ; ■ 

Developing the coiuKXiivc >ysicni to suppoH subcotutaci operation (e.g., how 
assessment will feed intoagency service operation) i ; 

Determining compliance with the L'ni/orm Guidelines on Employee Selection .... [1 

Putting the subcontract program in place I ; 

Regularizing ongoing operation and troubic shooting \\ 

Implementing revieV-atidevaluation activities i i 



Steps in Program Development 

Setting objectives and relating these to agency services and mission. Just as set- 
ling a.sse.ssinent ?oals is important for individual clienls, so setting overall 
goals for what assessment is expected to accomplish will help a program to 
develop in ways that maximize the efficiency of the agencyVs service delivery 
efforts. Also, establishing the objectives that the assessment can be expected to 
achieve helps determine how much the assessmei't should cost and how much 
assessment is enough. These objectives, in turn, should be related to the sec- 
ond step (below). 

In-house vs. subcontracted assessment program* For some employment and 
training agencies, this decision may be mandated b> regulations under which 
ihe agency operates, or tht*ie may be a completely free hand in deciding 
whether to maintain asssessment on the inside or contract it out. Cost may be 
the determining factor or avr^ilabiliiy of in-house staff to undertake assess- 
ment. For example, if an employment and training organization operates 
under the umbrella of a government agency (state or local) and if there is a hir- 
ing free/c on personnel, it simply may not be possible to bring in qualified 
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assessment statt to run the assessment program. Under such circumstances, 
t^ather than compromising the goals and objectives of assessment in order to 

ldeoltir» , ""'t^'u '''' ^'^^ ^'^^ ^» ^hc operation 

adequately . u may be better to contract the program out-if that is otherwise 
a leasible alicrnaiive. 



!^oJ^'"' theplannmn checklist for in-house 

1**3'!''' ^""""^^ ^'^^^ ^"'"^ Prevision what 

resources are available to support assessment: staff with some background in 
assessment, or mterest in acquiring it; dollar resources either within the agency 
IvmTi h """f a new funding proposal (see below); physical 

:i:;;ad;^ett i^to i;.^^^"^^""^ 

HoH does assessment fit In? Assessment doesn't happen in a vacuum. Assess- 
ment has to fit m to some degree with the rest the services the agency pro- 
vides, wtth tts relationships to local employers and to other .service agencies 
Time may be an especially critical variable here: If a.ssessment takes too long' 
employment options or training options through other service agencies may t 

ent nr^'L'^^^ \" "''"'"'^ ^^'^ ^ '^''^^ "^^^^ assess- 

mcnt program tor selected clients. 

Creating a planning committee. Staff who will have responsibility for assess- 
ment and personnel whose work functions will be affected by whatever assess- 
ment provides should be represented on this planning committee. In a large 
agency, the committee might be six to ten persons. In a smaller agency, two or 
three members might be sufficient. The point is to have represented all those 
Who ultimately will be affected by the assessment program. I might be elpf 

nu > nwh"' r ""^ ^'"^""^ °" committee lo provide in- 

put trorti the client perspective (in some .situations, this may not be feasible 
e.g.. It there is strong resistance to the idea by the agency director) 
nr.t ^^^ 'Tn^ "P^^-'^"y '"iPortant when a formal as.sessment 

er hi', n ^^^^ .'"f "'^.^ ^" age.tcy for the first time. There can be consid- 
c able psycho..-ical resistance among staff to a new assessment system-it 

nif Z "•'"''''^ ^\sc^^■h,r,. Staff participation in plan- 

r . g the a.ssessment program can pron.ot. program acceptance; people are 
much less likely, to resist new programs if they ha^e had a hand in creating 
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Key per^ions* Although the above-mentioned coiiiinittee may play a critical 
role mphrminji tor asscsMiient, in the last analysis, one or two individuals will 
likely be primarily responsible for actually implementing the assessment pro- 
gram once the planning has been completed. The key person or persons need 
to be clearly identified and adequate authority given to them so that they can 
do the job of creating the as.sessment program without undue interference 
either from other staff or from bureaucratic red tape. 

Staff involvemeiii. In addition to the planning vommittee's activities, certain 
aspects of assessment program developmenf can proceed best if there is an 
even wider staff involvenient in the development process. For example, when a 
planning committee has created the basic design for an assessment program, 
this design could be transcribed on paper in the form of a short planning 
repoit or document and then ciiculated for review by the entire staff of the 
emplovment and training agency. In general, the broader the review, the more 
useful the suggestions that will be obtained and the more built-in acceptance 
for the program that will result. .Another approach may be to hold a staff con- 
ference at which the basic assessment program design could be presented, with 
time allotted for qucstionv and answer^and for review of the program design. 

Information search. The appendix presents background required for this step. 

KeviewinK identified techhiques. Once an agency has decided what it wants to 
accomplish with assessment and has formulated a number of alternative 
approaches to meeting iliose objectives, the basic question becomes one of 
how to review available options for assessment techniques and then make deci- 
sions. One initial step is to develop criteria the particular program of assess- 
merit should meet. Wfiat are the cost limits? What are the time limits? What 
are the client characteristics? What are the staff capabilities in regard to assess- 
mcin? These all mav set hmits that will help to weed out certain identified 
assessment approaclies as not feasible for a given situation. 

( onsuitanis on assessment. In some cases, despite the presence of qualified 
people withm the agencv to help in developing an assessment program, the 
task o\ settmg in place a complex package of assessniem services may require 
outside consultation. If an av!encv determines that such consultation is needed, 
care must be taken to select individuals who can provide the expertise required, 
l or esample. professors in local college or university departn^MUs of psychol- 
ogy or education oiien mav liave the required background in psychometrics 
but lack practical expenetice tn employment and training services. By the same 
token, individuals who are \\\^\\\\ knowledgeable about employment and train- 
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ing issues vU'tcii do tioi uiidcrsidiid etunigh about psychological asscssinent to 
be able to contribute good input in that area. 

A good wa\ to start hunting tor a consultant is to check witli another 
eiiipUnment and training program that has pivMously established an assess- 
ment progtaiii. Much usetui input can be provided about problems encoun- 
tered and solutions obtained, botli in terms ot selecting a consultant and often 
on a whole range ot assessment program development problems. In some 
eases, the best consultant to retain may be the leader of the assessment pro- 
gram dcxelopment eftvnt at another agency. 

PackaKt^d leehniqutfs or programs. Iwo immediate options for "ready-made" 
special assessment techniques for the severely disadvantaged are the state 
Iniployment Service and commercial test developers. Volume II and its appen- 
dices provide specific access information. An impo. tant general principle to 
remember is that any organi/ation having only one or a small number of 
assessment techniques to recommend for a particular agency is likely to do just 
that recommend wliat they have available! The reality is that sellers of com- 
mercial assessment devices may undertake quite a "hard sell" in order to get 
an agency to buy what thev have to offer. This may include making unrealistic 
claims about the betietits to be expected from assessment, minimi/ing po.ssible 
operating problems .md sometimes dvmir^rading other available options. 

Moreover, many distributors o! assessment materials in the commercial sec- 
toi are not thenisehes psvchometrically sophisticated and they are usually not 
the creatois ot the material they are trying to market. They are in business to 
make .i profit by selling assessment tools or systems that someone else has 
cieateil. 

All ot this is not to say that an agency cannot safely purchase commercial 
.issessinent instruments -lai trom it. In many cases, a commercially available 
assessment ..n package may be a correct choice to meet a particular need, 
and valuable technical assistance may be available for its implementation. The 
availabilttv of training and maimenanee services is, in fact, one of the princi- 
pal advantages to some commercial systems. 

Hv (he same token, state I niploynient Services provide several Ussessmeiit 
devices designed e\press|\ for severely disadvantaged clients. I. ike the com- 
meicial lesi marketers. I mployment Service personnel are apt to recommend 
vvhai ihcv have on hand. However, they also have training and follow-up ser- 
vices available, phis rhe kev advantage of a very large, ongoing research pro- 
giam tot generating validiiv. reliability and normative data regarding these 
;;^trunienis. 
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DevelopinK a technique or proKram. In some cases, particularly if the purpose 
of assessment is quite simple, a brief form or questionnaire may be sufficient 
to derive the information needed. Staff of an agency, especially those with 
some prior experience in psychological assessment, may be quite capable of 
producing such a form on their own. Several of the Assessment Program 
Descriptions in Section 11 make clear that this can be done. 

In special circumstances, large agencies with equally large budgets for 
asses.sment might conmiission the development of an assessment technique or 
package by an individual or organization having appropriate capabilities. 
However, this option is often beyond the means of most employment and 
training agencies, which have limited budgets for assessment activities. 

Also, there are many dangers in creating one's own technique or program 
unless sufficient resources (including required expertise in psychometrics) are 
available for that task. Not the least among the dangers of this approach is 
that of legal liability, since "roll your own" techniques developed with little 
attention to validity or reliabUity requirements may be highly suspect. 

Adaplaiion. Many assessment techniques may not be usable directly in a given 
employment and training agency setting without appropriate adaptation. 
Again, rtlvicnce to the descriptions in Section 11 shows how adaptation has 
taken place in some real-life settings. Assessment program developers may 
need to use the basic criteria for evaluating assessment techniques pre.sented in 
Section 1 when determining whether an asse.ssment device or .system they arc 
thinking of adopting in fact needs to be adapted. For e.\ample, some users of 
the COATS system discussed in the second volume have found that not all 
components in this commercially developed, multifaceted assessment 
approach are relevant to their needs. Thus, adaptation may consist of simply 
deleting one or more C'O.ATS co'nponents. However, adaptation is a process 
that proceeds at the adaptor's peril unless care is taken to .see that there is 
integrity to what remains and that the information lost by eliminating or 
changing certain portions is information the agency can afford to lose. 

Compliance »ilh the i niform Guidelines. In the past, employment and train- 
ing agencies have had relatively little interference from government agencies 
Nvith regard to client assessmert. That situation is now changing. The Mark 
Battle .-Xssociates study in Section II was commissioned becau.se the Justice 
Department was concerned with the possible adverse effect of as.se.ssment pro- 
cedures on disadvantaged clients of employment and training agencies. The 
writer also is aware of Congressional investigations into as.se.ssment practices 
used bv Cl- I A prime sponsors. F inally, lawsuits charging discrimination are a 
reality that an increasing number of local agencies may face, 

•^6 



An> assessment program, therefore, must he developed with full cognl/ance 
of what the law is and what liability might result from selecting certain options 
tor assessment. Program developers should begin by reading the Uniform 
iiuulelines on tmp/owe Sfiecfion Proci-dures (Federai Kefiisier, 1978. Vol. 43, 
No. 166). C onsultation may also be available from state L-mployment Services! 
the II..S. l inployment Service or the Office of F ederal C ontract C ompliance.' 
An especially valuable step may be to consult with other employment and 
training agencies to discover how they have dealt with the issue of liability and 
how successful these strategies have been. 



DetermininK ♦■""^l'*- Once a basic operating plan for the ysse.ssmeni program 
has been developed, it is important to sketch out in more detail what it will cost 
in terms of materials, training of staff, facilities needed and associated Indirect 
costs (e.g., transportation of clients to a different physical site for a.sse.s.s- 
ment-a step to be avoided if possible). Administrators of the employment 
and ttaining agency may find it desirable to require that those planning assess- 
tiient prepare an overall cost report that estimates the cost of asse.ssment per 
client, broken down according to basic categories (cost of materials, staff 
time. etc.). 



Kormali/inj{ overall plan. After both a technical plan and cost plan for the 
assessment program have been reviewed by appropriate agency staff, an over- 
all plan can be put together, perhaps in a written report. This will serve as the 
"Bible" for operationali/ing the assessment program. 



S«lectinK '*lat't'- A factor sometimes neglected in selecting among existing staff, 
or hiring new people, with respect to an assessment program is whether the 
individual in question has an interest in client assessment. Individuals with 
backgrounds in psychology or education may be the most likely to have such 
interest and perhaps to have relevant technical skills as well. In any case, selec- 
tion needs to be on the basis of competence in assessment, or potential for 
learning such skills, which is partly a function of interest or motivation. 



IraininK '*«aft'. Depending upon what in-house expertise is available, training 
ot staft to administer and use assessment tools may be conducted by agency 
personnel or ntay require outside training activ ities. Local colleges or universi- 
ties may be contacted regarding available courses in psychometrics, and. in 
some cases, training may be provided through either the Hmployment Service 
or a commercial test vendor. .Mso. consultant*, retained for program develop- 
ment purposes otten may have the skills to conduct staff training sessions. 
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Validation efforts. }\iriicularl\ after reading the Cmform Guidelines, assess- 
ment program developers \\\\\ be painfully auare of the need for gathering 
data regarding \\v: impact of the assessment program. As a very beginning 
effort, routine record keeping should inchide entering both a summary of 
assessment resuhs and a summarv of what happened to the client following 
assessment. Some documentation, even if in the form of ise notes, should be 
provided regardnig whether and how assessment results were used. It is beyond 
the scope of this \ fanuiil to present a detailed strategy for developing a valida- 
tion siudv; an expert consultant mav be needed tor that purpose. 

OtiKoiiiK operation and trouble-shoo<5nj{. C learly, no newly developed assess- 
mem program, or one tliai has undergone major revision, will operate trouble- 
tree. In point of tact, the changing demands and changing client mix in most 
emplovment and training agencv settings means that there ar^; likely to be 
oporaiing problems on a regular basis. I he assessment syste-.n that takes this 
reahiv into account, and provides a routine procedure for ' de-bugging,'' will 
be mil more likelv to operate successfully over time. 

K\aUiation ind proKram improwmcnt. Whatever the administiative require- 
ments With respect to ev^iluaiion (e.g., to submit data for continued CliTA 
funding), evaluation also needs to be designed so that results can be used to 
suggest improvements ni program operations, hvaluation reports that disap- 
pear into administrative files, never to be seen again, serve Httle useful purpose 
(oihei than to assure that a program will continue to operate, but without pos- 
sihlv mucli-needed improvements). 

Presenting assessment programs in funding proposals. Assessment often is 
neglected in proposals tor tunding of employment and training agencies. Staff 
who pupare sucli proposals frequently are unsophisticated about assessment, 
Iluis. (lie assessnient program may be described *oo sketchily or in highly 
teclinical language if tlie proposal section is written by the resident ''expert on 
assessment " \ itlier uav, the result mav he difficulty in getting the assessment 
component apprvAed; in some cases, ilicre may even be a loss of funds for 
assessment because tliose reviewing the proposal did not understand what was 
being said I 

Some snnple suggestions tot improving the quality of :he sections in funding 
pioposals Hull describe an agencv \ assessment prc^gram areas follows: 

I. I he dcNcnpiion o\ Hie assessment program sliould begin with a clear state- 
ment o\ the r/Mvs o\ assessniem. Wliat, in simple I-nglish, is tlie assess- 
ment proLMam iniended to acconiplisli? Why will clients be better served 
with aNsi'SNUient ttiaii v\iihout it? 
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2. In the case of proposals lo funding agencies other than CHTA (but where 
the employmeni and training agency submitting the proposal does have 
responsibihtv to C'l* l A), it should be made clear that systematic assessment 
activities by agencies receiving C*IiTA funds are naw strongly encouraged 
by the i \ W legislation, so there is less tree choice about v^hethcr or not to 
have assessment at all. 

3. Components of the assessment program should be presented in summary 
t\)rm— again, in understandable I-nglish, not technical jargon. A chart or 
grapli outlining the basic steps in the assessment program might be especial- 
ly helpful (the basic elements chart presented earlier in this section could be 
used as a starting point for creating such a graphic aid to understanding). 

4. Having the section of a proposal concerning assessment reviewed for clarity 
and completeness bv someon<e whose background represents as closely as 
possible the background of those who will be judging the proposal would 
also be verv helpful. 

Vhe tollow'ifiM \uii\ie\(um\ are rele\an( to (he plunninii chvckUsi for suhcon- 
(rue fed i^ro^rams: 

Defining needs. See the paragraph on setting objectives above. 

(*:mHidennK alternatives. The state c'mployment Service can constitute a viable 
option for assessment subcontracting for many employment and training 
agencies. There is the adviMitai^e of access to assessment techniques developed 
by the T.S. I mplovnient Service and to USFSN large ongoing research pro- 
gram. In manv states, the liniploynieni Service m. y have had more experience 
assesNing clients than any other available organization, l-ducaiional institu- 
tions, such as a local university or college having appropriate testing personnel 
and programs or private nonprofit or for-profit organizations providing 
assessment services, also ^an be investigated. 

Perhaps the most important consideration here is to check out experiences 
o\ other local agencies in receiving assessment services from the various alter- 
native providers. I ven agencies serving other than employmeni and training 
populaiuMis may provide vahiable information by simply reviewing what their 
expeiK-nces [lavc been. (What has been the cost and impact on service of the 
suhcmuracted assessment program? What operating problems have arisen? 
How quickly have they been solved?) 

Makinv: contact. I lieie is probably no substiiuie for face-to-face meetings with 
persons representing each maior alternative resource for assessment an agency 
IS v'onsidcnng in oulci \ o learn what services -ach can provide and how . 
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t.vMluiilinK the «liernaii*cs. Soe the scctio.i ab.nc about considering alter- 
natises tor assesMnent. A particularly important couiiiieration is learning 
whether the subcontractor has onl> one assessment system available or could 
tailor one to tlK needs of the giu-n einpK>>men; and iraining agency. 

I)elerniininj{ costs. This should include costs to the cmplovment and training 
agencN itself that arc not included in estimates provided bv the potential sub- 
contractor, e.g.. tor time spent b> staff making input to the assessor.s. reading 
repoits ot results, etc. 

^^rllinK the subconlraci. V\ hatexer legal structure mav be required, it is impor- 
tant to get in urmng uhat uill be the responsibilities of both the employment 
and training agencN and the subcontractor with respect to assessment. Who i.s 
to do what, who IS to pay for it. and whose responsibility is it if something goes 
wrong? (.etting such requirements down on paper in advance of establishing 
the program -mav avoid man> nnsunderstandings and real problems later on. 

De^dopioK connectUe s>stem. How results of a subcontracted program will 
teed into the contracting .igenc>'s overall service process is important. When 
will reports be <Mnere(n What provisions will there be for counselors to get 
access to those whv> prepared the assessment reports if there are questions 
about the results and their implications? 

< ompliance. pultinK subconiraclinR program in place. ongoinK operation, 
re*ieH and evaluation. See discussion above." 

Applying the Checklist 

l or empUnment and training agency staff ready to begin planning a new or 
substamialK revised assessment system, it may be helpful to obtain some gen- 
eral background knowledge on organizational change and how to implemcm 
It. I he most comprehensive reference work now in existence on this subject is 
Put I ma KnouU'dni' to L \e: A Distillation of the Literature Regarditif; Knowl- 
edve Iranster and Chan^ie. published by the Human Interaction Research 
Insiituie HI An excellent synopsis of important principles of knowledge 

iranstei and organizational change is Cilaser ( 1973). Individuals or committees 
in charge of developing an assessmcm program may wish to include in their 
planning process some of the strategies these two works suggest on implemen- 

•l.it.>fm.ii,..n ,>n h,.« t,. ,.ht..>n .opv ,>t ihis publuMtum. ^hi.h is lu>^v hem)! rovi .-d tor a 1980 
V\..iKl I .liiu.ii. m.iv K- ..ht.iincJ hv ^vikiiij; u> the tiiMiiua- .11 lOXXV W iKhirc Houli-vard Smic 
11:0. I .'s \ii.!oli-N. (. A >AH):4 
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liiu» piograins of change, f or example, Purrinfi Knowledge ro Use contains 
some concrete reconnnendations about how to diagnose an agency's readiness 
to accept the change and then how to deal with problems of staff resistance if 
iliey aie found to exi^t. There are suggestions in both works about techniques 
for enhancing acceptance of change, such as trying out the new program on a 
trial or **piloi" basis before unplemeniing it full-scale. 
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A. I Ncllii!?! \- PrnfiU ihta, oudme for o in\elin}i model and reami- 

nwndiitiuns tor iruminn Pha^c 11 Report. Brainirec, M.A: .A. 1 . Nellum & Associ- 
ates, \^r4 

Backer, 1. V/tv/zin/v usst'ss/z/v/ ihe disadvania^ed tn manpower proj^rams: a 
rcvten undanalvsis. R\I) I inJingN No. U. Washington. D.C .: U.S. Departniem of 
I abor, 1^72. 

Brmu>, I /make ami usst'ss/wi7fr- CKIA programs models. Washington. D.C: U.S. 
<io\eiiwiieiu Prmimg Otf ice, 19^8. 

Cila f, \ M. Knowledge iranster and institutional change. Professtonal Psvcholo^v 
197 V 4. 4.U 444. ■ " ' 

Mark Battle ANSociaios XiiaUMN ot data on testing fro'n il I A prime sponsor-.. Wash- 
nigton. D.C".: .\uihor, l^'h. 

Puffim: kmiwlcdue lo use \ dtsiillanon of (he huraturv re^ardtn^i kmnvlcd^i- transfer 
and change 1 on .XiigelcN: Human Interaction Research Institute. 1976. 
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APPKNIMX 

Resources for Assessnient Program Developers 



Ot the many needs ernploviueni and iraiiiiiig agency siatT have when develop- 
ing an assessineni program, none is more pressing - and. particularly tor those 
who do not have a tornial training background in psvchological testing, none 
more frustrating - tlian the need tor information sources regarding assess- 
ment. The purpose ot this appendix is to provide a foundation or "jumping- 
off place'' for conducting: an information search-and-retrieval process in the 
field of client assessment. 

in order to know what information is required, the purposes and basic struc- 
ture of the assessment program must first be defined (decisions may or may 
not have been made yet about /)<yrm7//<^r paper-and-pencil tests or work sample 
systems thai might be selected for use). Some thoughts about the possible 
alternatives ought to be available, though, and the information search can 
begin vvitli these. 

The general reference most likely to be of nclp to as.sessment program dcvel- 
opers IS the writer's .1 Direaory of In/ormunon on Tests (Backer, 1977)* This 
monograph provides a directory of books, other print materials, and informa- 
tion systems that mav be useful in the search for test-related information. ALso 
included are some siraiogicN for mounting an information search. 

As everyone who Uses or develops psychological measuring instruments 
knows, the information explosion has reached educational and psychological 
testing. Surpnsinglv. even researchers who have dealt with measurement prob- 
lems tor manv vears ma> be relatively inexpert when it comes to information- 
retrieval skilN in this area. I herefore, part of the mission Directory was 
to increase professional capabilities in utilization of existing resources on edu- 
cational and psvchological measures. Although the Directory was not devel- 
oped primariK lor emplovment and training agency assessment program 
developers, it is vMguni/ed in a way that will permit relatively easy adaptation 
of Its information lor iliat [Hupose. 

I lie mam caiegones of information included in the /)/>m(>n' are as follows: 

Pnnt materials - /»(.mAn. nHimal articles and nwean h reports on the suhjeei of edu^ 
cafttinal anil /»mi /?,»/, .j^'/.w/ meastircmcm. i.ach cntrv includes a complete refer- 
tv/i citatum antl a hru'f \ i nopsi\ of !he document in quesiion. 



I Ik- ..MhpKf.- fvi,-f,-n.v -h!. pijhlu.iiton is Ihoiius I KuKci . 1 />/m ot Intonnutmn an 
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Information svstems-descnpnons oj orimruzumm pnnidinM tnfonnatton on tests 

and. m some ivses, aeiuul eopies of assessment msiruments 
Supptemtntary sourcts^mdexes to fmhltshed hteruture. nmuztne^ and journals 

providm)^ reviews mcludtna tnformafion in (he assessment area and informauon 

systems that ha\e a retne\al eapahduv hut are not exptwulv destined for retrieval 

on the suh/eet of assessment 

Alphaheii/cd indexes are priuided ai the end ot; the Directory to tacilitate 
easy access to its contents. 

Assessment program developers may wish to i^htain a copy of this Directory 
for their persiinal libraries. Several excerpts given here describe the most criti- 
cal published volumes of information on psychological testing that might be of 
interest to employment and training agency staff: 

Butos. Oscar K. 

lests mprtnt. Highland Park. N..I. : (irvphon Press. 1%!. 1974{Vols. 1, II). 

Reading tests and reviev%s. Highland Park. N.J.: Cirvphon Press, 1968 1975 
(Vols 1, II). 

Personahtv tests Highland Park. N..I.; Cirvphon Press, 1970, 1975 (Vols. 1, II). 

I'he seventh mental measurements yearbook. Highland Park, N.J.: Cirvphon 
Press, 1972 

Imelhiience tests and reviews. Highland Park, N.J.: Ciryphon Press, 1975. 

Vocational tests and reviews Highland Park, N.J.: Ciryphon Press, 1975. 

tn^h.sh tests and reviews. Highland Park, N.J.: Ciryphon Press, 1975. 

i oreiKnlammaiie tests and reviews. Highland Park, N. J.: Ciryphon Pre.ss, 1975. 

Mathematics tests und reviews. Highland Park, N.J.: Cirvphon Press, 1975. 

Saence tests and reviews Highland Park, N.J.: Ciryphon Press, 1975. 

Soiiul studies tests and reviews. Highland F>ark, N. J.: Ciryphon Press, 1^75. 

Iheeoihih mental measurements yearbook. Highland Park, N..I.: Cirvphon Press 
19"S. 

I hose Biiros puhlu-aiions are perhaps ihe best-knov\n reference sources for 
livanng various ivpcs of measuring instruments. The eight editions of the Men- 
tal Measurements Yearbooks f.MMY), supplemented by monographs on specific 
measurement topu's. provide comprehensive information on and critical 
reviews ol hundreds of measuring and data-collecting devices. Infor/naiion on 
Ihe instruments covered includes the following: anticipated subject populations; 
availahihtv of manuals, normative data, alternate forms and scoiing services; 
time required to administer the instrument: where it may be obtained; cost of 
the instrument, a list of available references about the instrument such as 
hooks, journal articles and dissertations; and (in many cases) a critical review of 
the tCNi bv one or more appropriatelv qualified professionals. The hi^hth Men- 
tal Measurements yearbook contains listings for 1,184 tests, H9H critical test 
rev tews bv 484 reviewers, 140 excerpts from test reviews that appeared in 29 



luiunaK. and \\4H\ rctcroncos \ot spoctic losls. Of iho tests iti this volume. 
X^()atotK•v^ ot revised Ihe A/.wm/ .V/t'.A»/-..m.'/,/s hw/>(;«As are cxiensiNely 
crosvmdexcd On autt.oi. test title and measurement topic area); all editions of 
tr>.c MM) and its amipanion volumes in peisonalits and leadint. arc similarly 
oi.uani/ed \v>o separaJc lists ot tests in print (1%1. 1^74) sei\e as masi-r 
.ndexes to the V/V/> laken together those 'volumes contain information on 
tnoNt published measuies 

Chun, ki laek. ( ohh. Sidnev. .V I rench. John K. P . I,. Manures for psvchohMicul 
.Ms.'ss/m7„ I ,ia<h'n.l,kH)or„ma/s,Hmvs ond ,har appl,vutions,\,m Arbor. 
Mkh ■ Institute tot Vvial Research. Umver. of Michigan. 1975. 
l iiirics toi the book vveie obtained through a search of 26 measurement-related 
loutnals 111 psvchologv and Mviolo^iv tor the lO-vear period l%()-W70 The vol- 
ume consists ot two niaior sections. I>rin,arv Kcfei\-nccs and Applications. In 
the pnmaiv references sociion. l.(KK) measures are cited: ttic applications sec- 
••on punides information about 6.600 instances in which the instruments 
described in the piiniarv references section have been used. liach primary refer- 
ence .elated to a particular test contains: a reference citation for a description 
ot the device, the measute\ title, and descriptors (kev words descriptive of the 
uisi.umem's content) l-.,ch entry in the applications section includes: a serial 
number used to identifv cacti test application instance: a reference (biblio- 
.«.apfiic emiv) denoting an article oi other publication that describes studies in 
vNhich the device has been used: a set of terms mdicatinj! the types of informa- 
tu'M in the aiti. le o. other publication cited: and otfier scales u.sed in the article 
ui othei publication listed. 

( omrev Andrew [ .. Hacker. Ihomas I-.. & (ilaser. l.dward M. A sourcebook for 
menial heahh measures I os Angeles: Human Interaction Research In.stitutc. 

Contains l.UK) abstracts, each of which provides information about a specific 
psvchoUvical test I IH- abstracts have been classified and grouped into 4S chrs- 
le.s beg.nninv. with Mcoholis,,, and ending with Vocational Tests, l-ach abstract 
.^ organ,/ed into two ma.or sections. The first presems identifving information- 
title of measure; source of the instrument (book, lournal article, mental health 
venter, and so fortti): and name of the author and address of source for further 
tnlormation I he second section is a 2(X)-.^(X)-word description, usually provid- 
n.g information on the following topics, the test's purpose, target population. 
.'dniiniMrauon time, number of items, tvpes of items and response mode used. 
avail.,ble reliabilitv and validitv data, and findings derived from anv major 
roseaivh applu.itioiis 

Kobmson. 1 1' . Mhanas.ou. K . A Head. K. H. Measures oeeurotmnal aimudes 
^'ul oceurai.onal eharaaensn. s. Ann Arbor. Mich.: Institute for Social 
Kesoar.h. 1-^6^ 

! hc second volume ot a th.ee volume collection of insirumem descriptions in 
the soc.o,>ohtKal domain. Measures ofOceupationa/ Aiiiiucles and Oecupa- 
Itonal Uiu 'Uensius. deals with measuies useful in studying the world of 
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\%ork. Some bO aiinudc sL-alcs arc reviewed, most of ihem dealing wiih job 
sanstaciiun, although other scales are included thai tap occupational values, 
leadership mvIcs and union-managemcn! altitudes. Three chapters examine 
measures ot iKcupaiional status and occupational similarity. The monograph 
also includes reviews ot surve> findings on job satisfaction and on status in- 
consistency. 

Many asscssnicni pri-graiii developers may find it of value to contact the 
V.RIC Clearinghouse .mi Tests, Measurement, and Evaluation (ERIC/TM), 
which is pacP of the massive Educational Resource.s Information Center 
(ERIC) sponsored by the U.S. Office of Education. The Clearinghouse pro- 
cesses documents related to assessment from test publishers, researchers and 
iitdividual researchers or research programs affiliated with a variety of practice 
settings. These documents are available through the ERIC Document Repro- 
duction Service, P.O. Box 190, Arlington, Virginia 22210. ERlC/TM also 
produces interpretive summaries, research reviews, and bibliographies on 
critical topics in measurement and can provide qualified users with customized 
computer searches of the ERIC data base. 

t rs Test Collevdon: The Educational Testing Service Test Collection is an 
extensive library of tests and other measurement devices. It was established to 
sci ve as an archive for testing and to provide current information on available 
tests and related services to persons engaged in research, advisory services, 
education and related activities. The collection presently compri.ses more than 
10,(XX) instruments. In addition to tests, files on American and foreign les*. 
publishers, scoring services and systems, state testing programs, published test 
reviews. and reference materials on measurement and evaluation are main- 
tained. 

Qualified persons, whether affiliated with Educational Testing Service or 
not, may have on-site access to the materials in the test collection. However, 
publishers' restrictions regarding a:ce^■s to test materials are carefully 
observed. PersiMis unable to visit the collection may direct specific questions to 
its staff by mail or telephone. 

Notification about recent additions to the collection and announcements of 
new tests and services are provided in News on Tesis, a newsletter published 
monthly except June and August and available on a subscription basis. New 
publishers, new references on measurement and evaluation, a calendar of con- 
ferences, references to test reviews, and notification of publisher changes are 
also included in this publication. 

Another regular publication of the ETS Test Collection is the Major U.S. 
Publishers of Stamianhzed Tests, a pamphlet listing publishers' addresses. In 



addition, a series ot annotated test bibliographies has been prepared and is 
available on request. 

Besides News on Fests and the publishers' directory. ETS Test Collection 
publications include the ft;i|owing bibliographies; 

Assessmeniof Teachers. Match 1977 

Aituudes Toward School and Schcoi Adjustment. CJrades4-6. May 1977 

Attitudes Toward School and School Adjustment. Cirades 7-12. March 
1973 (hi) 083 323) 

Criterion- Referenced Measures. C5r;u os4-6. October 1978 

Measures o/ Self-Concept, Cirades 4-6. February 1977 

Readinsi TtMs. Cirade',4-6. June 1977 

Readint' Tests. 7-16 aiid Adults, June 19*,'? 

Self-Concept Measures. Grade 6 and Above. March /977 

tests for EduL ationully Disadvantaged Adults I '..ci uarv i 977 

For readers wishing to obtain a general background in standards of excel- 
lence lor assessment measures, one particularly valuable reference is the 
American Psychological Assi>ciat ion's Standards for Educational and Psycho- 
logical rf.v/M Washington. DC: American Psychological Association. 1974). 
A directory for information-retrieval services, including ERIC and a large 
number of other systems that may have some capability for retrieving informa- 
tion about assessment measures, is provided in the Human ls>»cract:on 
Research Institute's publication Information Sources and How to Use fhem 
(I OS Angdes: Human Interaction Research Institute, 1975)* The reader is 
referred to Section HI for additional suggestions regarding how to access 
information about assess.-nent techniques already in use in employment a-id 
training agencies. 

A description of the Materials Development Center and its publications, tlv. 
major resource for up-to-dato information regardinj; work sample sysien.s, i-. 
presented in Appendix B of Volume 11 along with oiher information ahoui 
resources tor assessment of the disabled. 

For a conceptual oserview of some of the issues involv-d in assessment of 
individuals from mmoritv groups, see Ronald J. Samuda. Psychological Test- 
ing of American Minorities: Issues and Consequences (New York: Huri.tr & 
Row. 1975). 



X ..p,CN,.| Intamwiun Sour,,: m.is bf I'hi.iiru-d tree ot vhargc b> untir,}! to iho Institute at 10889 
NVilshiic H«>ulf\.iui. Suite 11:0. 1 »s Angeles. I A y(X):4. 
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